Recording medium, playback apparatus and method, recording method, and computer-readable program

ABSTRACT

A BD-ROM stores a video stream and a graphics stream. The video stream represents a moving picture. The graphics stream is used for overlaying a multi-page menu on the moving picture, and includes interactive control information (ICS) and graphics data (ODS) used for generating the multi page menu. A PTS attached to a PES packet containing the interactive control information shows timing for the first presentation of a main page the multi-page menu in accordance with the proceeding of the video stream playback. In addition, the interactive control information includes information (selection_time_out_pts) showing a timeout upon which a button on a page of the multi-page menu is automatically activated, and information (user_time_out_duration) showing a timeout upon which a sub-page of the multi-page menu is automatically removed.

TECHNICAL FIELD

The present invention relates to a field of user interface control. Moreparticularly, the present invention relates to improvements in recordingmediums used for consumer movie distribution, as well as in consumerplayback apparatuses having a user interface for playback of therecording mediums.

BACKGROUND ART

A user interface using a menu structure is an essential function forreceiving user operations, such as selection of languages and chaptersfor playback.

When a movie offers a number of user-selectable options such aschapters, there may be a case where all the options can not be presentedon a single screen display. Accordingly, there is a demand for amulti-page menu with which multiple pages are sequentially presented inresponse to user operations, so that numerous options are appropriatelypresented on respective pages of the menu.

Unfortunately, however, if too many pages are displayed, those pagesoccupy a large portion of the screen, thereby interfering with viewingthe main movie also displayed on the screen. Such a problem can beavoided if a user repeatedly makes operations for removing each pagethat is no longer of his interest. However, the need to make suchtroublesome operations over and over may be another factor bothering theuser watching the movie.

In addition, the interactive control that a DVD provides has thefollowing disadvantage. For each menu page to be displayed, an accessneeds to be made to the DVD to read page information. Because of theaccess, the menu page can not be displayed without interruption to videoplayback. The interruption makes it difficult for the user toconcentrate on, and enjoy the movie.

DISCLOSURE OF THE INVENTION

The present invention aims to provide a recording medium supportingpresentation of multi-page menus which will not interfere with the videoplayback.

To achieve the above aim, the present invention provides a recordingmedium having a video stream and a graphics stream recorded thereon. Thevideo stream represents a moving picture made up of a plurality ofpictures. The graphic stream is used for overlaying a multi-page menu onthe moving picture, and includes interactive control information andgraphics data. The graphics data is used for generating the multi-pagemenu. The interactive control information includes time information usedfor controlling behavior of the multi-page menu in accordance with aplayback proceeding of the video stream.

According to the recording medium having the above structure, theinteractive control information multiplexed in the graphics streamincludes control information. The control information defines themulti-page menu behavior in accordance with the proceeding of the movieplayback. This structure makes it possible to automatically present anumber of pages during the time the movie playback is showing scenery,for example. In addition, it is also possible to automatically removethe pages from the screen when the movie playback reaches a climaxscene. With such control, the multi-page menu is presented withoutinterfering with viewing the movie playback. Consequently, the user canenjoy the convenience provided by the multi-page menus, while watchingthe movie playback without obstructive display.

In addition, the information used for controlling the multi-page menubehavior is included in the graphics stream, and the graphics stream ismultiplexed with the video stream. This structure allows a playbackapparatus to read the information needed for presentation of themulti-page menu, without interrupting the video stream reading. Thus,pages of the multi-page menu are smoothly presented in synchronizationwith the video frame rate.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a view showing a usage pattern of a recording medium accordingto the present invention;

FIG. 2 is a view showing an example structure of a BD-ROM;

FIG. 3 is a view schematically showing the structure of an AV Clip;

FIG. 4A is a view showing the structure of an interactive graphicsstream;

FIG. 4B is a view showing PES packets carrying functional segments;

FIG. 5 is a view showing the logical structure defined by various typesof functional segments;

FIG. 6 is a view showing the time axis of AV clip playback to which aDS_(n) is allocated;

FIGS. 7A and 7B are views showing the relations betweeninteractive_composition structures and ICSs;

FIG. 8 is a view showing the internal structure of an ICS;

FIG. 9 is a view showing the processing performed on an ICS in the casewhere a stream_model field shows a “Multiplexed” type;

FIG. 10 is a view showing the processing performed on an ICS in the casewhere the stream_model field shows a “Preload” type;

FIG. 11 is a timing chart illustrating the functions ofselection_time_out_pts and composition_time_out_pts fields in an ICSbelonging to the DS_(n) in the case where the DS_(n) is an Epoch StartDS (n=1);

FIG. 12A is a view showing the state transition of a Multi-page menuwhen a Pop-Up U/I is used;

FIG. 12B is a view showing the state transition of the Multi-page menuwhen an Always-On U/I is used;

FIG. 13 is the display scenario incorporating the Multi-Page menu;

FIGS. 14A-14C show a display example defined by theselection_time_out_pts;

FIGS. 15A-15D show a display example defined by theuser_time_out_duration;

FIG. 16 is a view showing the internal structure of page informationassociated with an arbitrary one of multiple pages (x^(th) page) of theMulti-Page Menu;

FIG. 17 is a view showing the state transition of buttons O-A, O-B, O-C,and O-D;

FIG. 18 is a view showing an example of the button_info descriptiondefining the button state transition shown in FIG. 17;

FIG. 19 is a view showing graphical images derived from ODSs 11-19;

FIG. 20 is a view showing an effect_sequence structure;

FIG. 21A is a view schematically illustrating how an effect_duration isdetermined;

FIG. 21B shows an equation for calculating the effect_duration;

FIG. 22 is a view showing a specific example of an in-effect animation;

FIG. 23 is a view showing an out-effect animation presented inaccordance with the window_width, window_height, and effect_durationfield values;

FIG. 24 is a view showing the internal structure of an arbitrarycomposition_object(i) in detail;

FIG. 25 is a view showing a specific example of an in-effect presentinga Right-Scroll animation effect;

FIG. 26 is a view showing an example of the composition_object (0)structure realizing the in-effect shown in FIG. 25;

FIG. 27 is a view showing an example of the composition_object(1)structure realizing the in-effect shown in FIG. 25;

FIG. 28 is a view showing an example of the composition_object(2)structure realizing the in-effect shown in FIG. 25;

FIG. 29 is a view showing an example of the composition_object(3)structure realizing the in-effect shown in FIG. 25;

FIG. 30 is a view showing the order of ODSs within the DS_(n);

FIG. 31 is a view showing how the order of ODSs within S-ODSs differsbetween the cases where the default_selected_button_id ref field is setto “00” and to the valid value specifying a button B;

FIG. 32A is a view showing a LASTODSPTS in the case where the defaultselected button is statically determined;

FIG. 32B is a view showing a LASTODSPTS in the case where the defaultselected button is dynamically determined;

FIG. 33A is a view showing a LASTODSPTS in the case where the DSincludes no ODS associated with the selected state;

FIG. 33B is a view showing a LASTODSPTS in the case where the DSincludes no ODS associated with the normal state;

FIG. 33C is a view showing a LASTODSPTS in the case where the DSincludes no ODS at all;

FIG. 34A is a view showing an algorithm for obtaining the DECODEDURATIONvalue from the value calculated by LASTODSPTS (DS_(n))−DTS (DS_(n)[ICS])and the value of PLANECLERATIME;

FIGS. 34B and 34C are views showing examples of how to calculate theDECODEDURATION value;

FIG. 35A is a view showing an equation for calculating the EFFECTTD;

FIG. 35B is a view showing an equation for calculating the PAGDEFAULTTDin the case where no in-effect is provided and the default selectedbutton is statically determined;

FIG. 35C is a view showing an equation for calculating thePAGENODEFAULTTD in the case where no in-effect is provided and thedefault selected button is dynamically determined;

FIG. 36 is a view showing an algorithm for selectively determining oneof EFFECTTD, PAGEDEFAULTTD, and PAGENODEFAULTTD as TRANSFERDURATION;

FIG. 37 is a view showing the internal structure of a playback apparatusaccording to the present invention;

FIG. 38A is a flowchart of “procedure when playback condition ischanged”, preformed with respect to a PSR 11;

FIG. 38B is a flowchart of “procedure when change is requested”,preformed with respect to the PSR 11;

FIG. 39A is a flowchart of “procedure when playback condition ischanged”, performed with respect to a PSR 10;

FIG. 39B is a flowchart of “procedure when change is requested”,performed with respect to the PSR 10;

FIG. 40 is a view illustrating the internal structure of an I-Graphicsdecoder 9;

FIG. 41 is a flowchart of the main routine of the procedure performed bya graphics controller 37;

FIG. 42 is a flowchart of the procedure for the first presentation;

FIG. 43 is a flowchart of the procedure for playback of an in-effectanimation;

FIG. 44 is a flowchart of the display update procedure for presentationbuttons in animation;

FIG. 45 is a flowchart of the procedure for executing a navigationcommand;

FIG. 46 shows the procedure for processing a UO;

FIG. 47 is a flowchart of the current button change procedure;

FIG. 48 is a flowchart of the procedure for auto-activation of a button;and

FIG. 49 is a flowchart of the manufacturing process of the BD-ROMsaccording to the first to fifth embodiments.

BEST MODE FOR CARRYING OUT THE INVENTION First Embodiment

Hereinafter, a description is given to an embodiment of a recordingmedium according to the present invention. First of all, a descriptionis given to usage of the recording medium, which is one form ofpracticing the present invention. FIG. 1 shows a usage pattern of therecording medium according to the present invention. In the figure, aBD-ROM 100 is are cording medium according to the present invention. TheBD-ROM 100 is used for providing movies to a home theater systemcomposed of a playback apparatus 200, a remote controller 300, and atelevision 400.

This concludes the description of the usage of recording mediumaccording to the present invention. Next, a description is given toproduction of the recording medium, which is another form of practicingthe present invention. The recording medium can be embodied by makingimprovements to an application layer of a BD-ROM. FIG. 2 shows anexample structure of the BD-ROM 100.

In the figure, the BD-ROM 100 is shown on the fourth level, and aBD-ROM's track is shown on the third level. The track is stretched outinto a straight line in the figure, although the track in practicespirals outwards from the center of the BD-ROM. The track is composed ofa lead-in area, a volume area, and a lead-out area. The volume area hasa layer model of a physical layer, a file system layer, and anapplication layer. The first level shows, in a directory structure, aformat of the application layer (application format) of the BD-ROM.

The first level shows the format of the application layer (applicationformat). In the figure, the BD-ROM has a ROOT directory, and a BDMVdirectory under the ROOT directory, and a STREAM directory under theBDMV directory.

The STREAM directory contains an AV clip and Sub clips. The AV clip andSub clips are contained in files in the STREAM directory, such as“00001.m2ts”, “00002.m2ts”, and “00003.m2ts”. Each file in the STREAMdirectory has a filename of a unified format, “xxxxx.m2ts” where x is anarbitrary integer.

<AV Clip Structure>

Now, a description is given to the AV clip. FIG. 3 is a viewschematically showing the structure of the AV Clip.

In the drawing, the middle level shows the AV clip. This AV clip can becreated as follows. The video stream made up of a plurality of videoframes (pictures pj1, pj2, pj3, . . . ) and the audio stream made up ofa plurality of audio frames both shown on the upper first level areseparately converted to PES packets shown on the upper second level, andfurther converted to TS packets shown on the upper third level.Likewise, the presentation graphics stream (hereinafter, “PG stream”)and the interactive graphics stream (hereinafter, “IG stream”) bothshown on the lower first level are separately converted to PES packetsshown on the lower second level, and further converted to TS packetsshown on the lower third level. These TS packets of the video, audio, PGand IG streams are multiplexed to form the AV clip.

The PG and IG streams are data in stream format used for compositinggraphics with each picture of the video stream. An entity of datacarried by the PG and IG streams for overlaying on a picture is referredto as a “graphics object”.

The PG stream is a graphics stream used for presentation of subtitlesusing the graphics objects.

The IG stream is a graphics stream used for interactive presentation ofthe graphics objects. This concludes the description of the AV clipstructure. Next, a description is given to the AV clip and the Subclips. The Sub clips are not multiplexed. Rather, the Sub clips arefiles exclusively containing the IG stream, audio stream, and text data.The IG stream stored as a Sub clip is loaded to a memory in advance forsynchronous playback with an AV clip. This operation of loading a Subclip to a memory in advance is referred to as “preloading”.

Next, a description is given to the IG stream. FIG. 4A is a view showingthe structure of the IG stream. On the first level, a string of TSpackets constituting the AV clip is shown. On the second level, a stringof PES packets constituting the graphics stream is shown. The PESpackets shown on the second level are formed by concatenating payloadsof TS packets having a predetermined PID within the TS packet stringshown on the first level. Note that no further description is given tothe PG stream because it is not the gist of the present invention.

On the third level, the structure of graphics stream is shown. Thegraphics stream is composed of functional segments that include an ICS(Interactive Composition Segment), a PDS (Palette Definition Segment),an ODS (Object Definition Segment), and an END (End of Display SetSegment). Of these functional segments, the ICS is a display compositionsegment, whereas the PDS, ODS, and END are definition segments. Eachfunctional segment is either in one-to-one or one-to-multiplecorrespondence with PES packets. That is, one functional segment isrecorded on the BD-ROM 100 after being converted to a single PES packet,or fragmented and converted to a plurality of PES packets.

Hereinafter, a description is given one by one to those functionalsegments.

The ICS is a functional segment used to define an interactive displaycomposition of graphics. One example of the interactive displaycomposition is a Multi-Page menu.

The ODS is a run-length encoded graphics object composed of a pluralityof pieces of run-length data. The run-length data expresses a pixelstring using a Pixel Code which shows a pixel value and a continuouslength of the pixel value. The Pixel Code is an 8-bit value showing oneof the values from 1 to 255. Through the use of this Pixel Code, therun-length data sets arbitrary 256 pixel colors out of full color(16,777,216 colors).

The PDS is a functional segment storing pallet data. The pallet datashows combinations of Pixel Codes of 1 to 255 and pixel values. Thepixel value used herein is made up of a red color difference component(Cr value), a blue color difference component (Cb value), a luminancecomponent (Y value), and a transparency (T value). Substituting a PixelCode of each piece of run-length data into a pixel value shown by thepalette data produces a color.

The END provides an explicit indication that the transmission offunctional segments is complete. The END is inserted into the graphicsstream as the last segment. This concludes the description of eachfunctional segment.

FIG. 4B is a view showing PES packets carrying functional segments. Asshown in the figure, each PES packet is composed of a packet header anda payload. The payload is an entity carrying a functional segment, andthe packet header carries a DTS and a PTS associated with the functionalsegment. Hereinafter, a DTS and a PTS in a packet header of a PES packetcarrying a functional segment are simply described as a DTS and a PTS ofthat functional segment.

These various types of functional segments define a logical structuresuch as the one shown in FIG. 5. In the drawing, the first level showsEpochs, the second level shows DSs (Display Sets), and the third levelshows types of the DSs. The functional segments shown on the third levelin FIG. 4A is shown on the fourth level in this figure.

An Epoch on the first level refers to one time unit of continuous memorymanagement on a time axis of the AV clip playback, and also refers to agroup of data allocated to that time unit. Memory mentioned hereinincludes a graphics plane for storing one screen of graphics objects andan object buffer for storing uncompressed graphics data. Continuousmemory management means that neither the graphics plane nor the objectbuffer is flushed throughout the Epoch, and that clearing and renderingof graphics are performed only within a predetermined rectangular areaon the graphics plane (to flush means to clear the entire graphics planeor the entire object buffer). The size and position of this rectangulararea are fixed throughout the Epoch. As long as clearing and renderingof graphics are performed within this fixed rectangular area on thegraphics plane, seamless playback is guaranteed. In other words, theEpoch is a time unit of seamless playback on the time axis of the AVclip playback. To change the graphics clearing/rendering area on thegraphics plane, it is necessary to define a point on the playback timeaxis at which the change is effected and to set a new Epoch from thepoint onward. In this case, the boundary between two Epochs is notseamless.

The seamless playback mentioned herein means that the clearing andrendering of graphics are each completed with a predetermined number ofvideo frames. In the case of IG stream, the number of video frames isfour to five. The number of video frames is determined based on theratio of the fixed rectangular area to the entire graphics plane and thetransfer rate between the object buffer and the graphics plane.

A DS on the second level is a group of functional segments included inthe graphics stream and provides one complete display composition. InFIG. 5, dashed lines hk1 show to which Epoch the DSs on the second levelbelong. The DS₁, DS₂, DS₃ . . . , and DS_(n) are shown to belong to thefirst Epoch on the first level.

The third level shows the types of DSs. The first DS in an Epoch is an“Epoch Start” DS. Each DS other then the first one in the Epoch iseither an “Acquisition Point” DS, a “Normal Case” DS, or an “EpochContinue” DS. Though the DSs are shown in the order of AcquisitionPoint, Normal Case, and Epoch Continue, this order is merely an exampleand the DSs may be arranged in a different order.

The Epoch Start DS provides a display effect “new display”, andindicates a start of a new Epoch. Thus, the Epoch Start DS contains allfunctional segments necessary for the next screen composition. The EpochStart DS is provided at a point to which a skip operation is likely tobe made, such as a start of a chapter in a movie.

The Acquisition Point DS provides a display effect “display refresh”,and is identical in contents to the preceding Epoch Start DS. TheAcquisition Point DS is not the start of the Epoch, but contains allfunctional segments necessary for the next screen composition.Therefore, graphics can be displayed reliably when playback is startedfrom the Acquisition Point DS. That is to say, the Acquisition Point DSenables a display composition to be made from a midpoint in the Epoch.

The Acquisition Point DS is provided at a point to which a skipoperation may be made, such as a point that may be designated by a timesearch. The time search is an operation of locating a playback pointcorresponding to a time input by a user in minutes/seconds. The timeinput is made in a relatively large unit such as ten minutes and tenseconds. Accordingly, the Acquisition Point DS is provided at a pointthat can be designated by a time search made in units of 10 minutes and10 seconds. By providing the Acquisition Point DS at such a point thatcan be designated by a time search, playback of the graphics stream canbe smoothly carried out when a time search is conducted.

The Normal Case DS provides a display effect “display update”, andcontains only a difference from the immediately preceding displaycomposition. For example, if a DS_(v) is identical in contents to animmediately preceding DS_(u) but different in display composition, theDS_(v) is a Normal Case DS which contains only an ICS or an ODS. Thiseliminates the need to provide overlapping ODSs in DSs, so that theamount of data stored on the BD-ROM 100 is reduced. Since the NormalCase DS contains the difference without the overlapping data, no displaycomposition can be displayed with the Normal Case DS alone.

The Epoch Continue DS indicates at the start of an AV clip that oneEpoch continues from a different AV clip. For example, a DS_(n) containsa composition state, field having a value indicating “Epoch Continue”.In this case, the DS_(n) belongs to the same Epoch as an immediatelypreceding DS_(n−1) even if the two DSs are on different AV clips. Sincethe DS_(n) and DS_(n−1) belong to the same Epoch, there will be noflushing of the graphics plane and of the object buffer even if there isa boundary of AV clips between the two DSs.

In FIG. 5, dashed lines kz1 show to which DS the functional segments onthe fourth level belong. Since the functional segments shown in FIGS. 4Aand 5 are the same, it is known that the functional segments shown inFIG. 4A belong to the Epoch Start DS. Functional segments belonging tothe Acquisition Point DS are identical to those belonging to the EpochStart DS. In addition, functional segments belonging to the Normal CaseDS are identical to those belonging to the Epoch Start DS, except thatsome of the functional segments are omitted in the Normal Case DS.

This concludes the description of the logical structure defined by thefunctional segments. Next, a description is given to the allocation ofDSs having such an ICS and ODSs, to the time axis of AV clip playback.Since the Epoch is a time unit of continuous memory management andincludes one or more DSs, it is important how those DSs are allocated tothe time axis of the AV clip playback. Note that the “time axis of theAV clip playback” refers to the time axis used for defining the decodetiming and playback timing of individual pictures multiplexed in the AVclip. The decode timing and playback timing are expressed in the timeaccuracy of 90 KHz. The DTS and PTS of an ICS or of an ODS within a DSshow the timing based on which the synchronous control is realized. DSsare allocated to the time axis of the AV clip playback by carrying outthe synchronous control in accordance with each DTS and PTS attached toan ICS and to an ODS.

Suppose an arbitrary one of DSs within an Epoch is denoted as a DS_(n),and that the DS_(n) is allocated to the time axis of the AV clipplayback based on the DTS and PTS set as shown in FIG. 6.

FIG. 6 shows the time axis of AV clip playback to which the DS_(n) isallocated. In the figure, the duration of DS_(n) starts at the timeshown by the value of DTS of the ICS belonging to the DS_(n) (DTS(DS_(n)[ICS])), and ends at the time shown by the value of PTS of theEND belonging to the DS_(n) (PTS(DS_(n)[ICS])). The display timing forthe first presentation of the DS_(n) is shown by the value of PTS of theICS(PTS(DS_(n)[ICS])). By adjusting the PTS (DS_(n)[ICS]) value tocorrespond to the display timing of a desired picture within the videostream, the first presentation of the DS_(n) will be synchronized withthe video stream.

The PTS(DS_(n)[ICS]) value is the sum of the time taken for decoding theODS (DECODEDURATION) and the time taken for transferring the decodedgraphics object (TRANSFERDURATION).

The decoding of ODS needed for the first presentation is completedwithin the DECODEDURATION. The duration mc1 shown in FIG. 6 is the timerequired for decoding an ODS_(m), an arbitrary one of ODSs within theDS_(n). The start of the decode duration mc1 is shown by the value ofDTS(ODS_(n)[ODS_(m)]), and the end of the decode duration mc1 is shownby the value of PTS (ODS_(n)[ODS_(m)]).

An Epoch is defined by allocating every ODS within the Epoch to the timeaxis AV playback as described above. This concludes the description ofthe allocation to the time axis of AV playback.

One feature of the first embodiment is to control the behavior of aMulti-page menu in accordance with the proceeding of AV playback. Thenovel structure realizing the feature resides in aninteractive_composition structure provided in the ICS. Hereinafter, adescription is given to the interactive_composition structure.

FIGS. 7A and 7B are views showing the relations between theinteractive_composition structures and the ICSs. Theinteractive_composition structure and the ICS may be in one-to-onecorrespondence as shown in FIG. 7A, or one-to-multiple correspondence asshown in FIG. 7B.

The one-to-one correspondence is applicable to the case where theinteractive_composition structure is small enough to be contained in oneICS.

The one-to-multiple correspondence is applicable to the case where theinteractive_composition structure is too large in size to be carried byone ICS and thus fragmented to be stored in multiple ICSs. Since theinteractive_composition structure can be fragmented to be stored inmultiple ICSs, there is no restriction imposed on the size ofinteractive_composition structure. The interactive_composition structuremay be as large as 512 Kbytes, 1 Mbyte, or any number of bytes. Asabove, one interactive_composition structure may correspond to multipleICSs. For the simplicity sake, however, a description hereinafter isgiven on precondition that the ICS and interactive_composition structureis in one-to-one correspondence.

FIG. 8 is a view showing the internal structure of an ICS. The ICScontains one interactive_composition structure or a fragment of oneinteractive_composition structure. As shown in the left part of FIG. 8,the ICS is composed of the following fields: “segment descriptor”showing the segment type “ICS”; “video_descriptor” showing the width andheight of the video plane in pixels and the frame rate of the associatedvideo stream; “composition_descriptor” showing the composition_state,which is information showing the type of DS to which the ICS belongs iswhether a Normal Case DS, an Acquisition Point DS, an Epoch Start DS, oran Effect_Sequence; and “interactive_composition data_fragment” showingthat whether the ICS is carrying an entire interactive_compositionstructure or a fragment of interactive_composition structure.

As indicated by an arrow cu1, FIG. 8 shows the internal structure of aninteractive_composition structure in detail. The interactive_compositionstructure is composed of the following fields:“interactive_composition_length”, “stream_model”,“user_interface_model”, “composition_time_out_pts”,“selection_time_out_pts”, “user_time_out_duration”, “page information(1), (2) . . . (i) . . . (number_of_page-1)” for each page of theMulti-Page menu available for presentation.

The interactive_composition_length field shows the length of theinteractive_composition structure.

The “stream_model” field shows the type of stream model of theassociated interactive_composition structure. The stream model shows howthe interactive_composition structure is stored on the BD-ROM and howthe data is to be processed by a buffer (referred to as a “compositionbuffer”) on a playback apparatus. Specifically, the stream model showswhether (i) the graphics stream is multiplexed with an AV clip andseparated to be loaded to the composition buffer or (ii) a graphicsstream is stored as a Sub clip in a separate file from the AV clip andpreloaded to the composition buffer. The stream_model field is providedin the interactive_composition structure because the composition bufferneeds to perform different processing depending on whether theinteractive_composition structure is preloaded or multiplexed. FIG. 9 isa view showing the processing performed on the “Multiplexed” ICS. In thefigure, the video stream multiplexed in the AV clip is shown on thefourth level, and the graphic stream multiplexed in the AV clip is shownon the third level. The graphics stream includes three DSs, which are aDS₁, a DS₂, and DS₈. The second level shows the proceeding of AVplayback. The first level shows contents stored in the compositionbuffer at different times of the AV playback. When the current playbackpoint reaches the beginning of DS₁ multiplexed in the AV clip, the DS₁is loaded to the composition buffer as indicated by an arrow pf1. As aresult, the composition buffer stores DS₁[ICS]. Then, when the currentplayback point reaches the beginning of DS₂ multiplexed in the AV clip,the DS₂ is loaded to the composition buffer as indicated by an arrowpf2. As a result, the composition buffer stores DS₂[ICS]. That is, inthe case where the stream_model is “Multiplexed”, there are more thanone interactive_composition structures. The playback apparatus discardsa currently stored interactive_composition structure and newly loadsanother interactive_composition structure residing at the currentplayback point. Thus, the composition buffer stores oneinteractive_composition structure at a time.

This discarding and loading operation is especially effective at thetime of a skip operation. When a skip operation is executed, theinteractive_composition structure associated with the current playbackpoint is no longer necessary. Thus, the playback apparatus discards theinteractive_composition structure currently on the composition buffer.As shown in FIG. 9, when a skip operation am1 is executed, thecomposition buffer discards the stored contents at the execution timingam2 of the skip operation. Then, the DS₈[ICS] residing on the skipdestination is loaded to the composition buffer as indicated by an arrowpf3.

As above, in the case of the “multiplexed” type stream model, thecomposition buffer needs to discard the stored interactive_compositionstructure as the AV playback proceeds. On the contrary, in the case ofthe “preload” type stream model, the composition buffer is not requiredto discard the stored interactive_composition structure at all. If thecomposition buffer discards the stored interactive_composition structureas the AV playback proceeds, the same interactive_composition structureas the discarded one needs to be loaded from the BD-ROM again. Thisuseless operation of re-loading the same interactive_compositionstructure results in an extra burden. FIG. 10 is a view showing thecontents of the composition buffer. FIG. 10 is similar to FIG. 9 withrespect to what is shown in each of the first to fourth level. Yet, FIG.10 is totally different from FIG. 9 with respect to the contents of thecomposition buffer stored at different times. Specifically, even if thecurrent playback point reaches where the DS₁ is multiplexed, the ICS ofthe DS₁ is not loaded to the composition buffer (indicated by the mark“X” in the figure).

Similarly, even if the current playback point reaches where the DS₂ ismultiplexed, the ICS of the DS₂ is not loaded to the composition buffer(indicated by the mark “X” in the figure). Similarly, the contentsstored in the composition buffer are retained rather than discarded,even if a skip operation is executed. The interactive_compositionstructure is provided with the stream_model field for allowing acontrolling subject (the graphics controller) to accurately judgewhether to discard the stored interactive_composition structure as theproceeding of AV playback or to retain the storedinteractive_composition structure irrespective of the proceeding of AVplayback. This concludes the description of the stream_model field.

Referring back to FIG. 8, the “user_interface_model” shows the type ofuser interface model to be used for the interactive_compositionstructure. The user_interface_model shows either “Always-On U/I” or“Pop-Up U/I”. When the Always-On U/I is used, a menu appears on screenand disappears in accordance with the proceeding of AV clip playback.When the Pop-Up U/I is used, a menu appears on screen and disappears inresponse to a user operation.

The composition_time_out_pts field shows the end time of an Epoch (EpochEND) to which the ICS belongs. The interactive control defined by theICS is no longer valid at the Epoch END. That is, a point of time shownby the composition_time_out_pts field is the end of valid interactivecontrol period.

The “selection_time_out_pts” field shows the button selectiontermination time. At the time shown by the selection_time_out_pts value,the button that is currently ins the selected is automaticallyactivated. Note that buttons are elements representing options in themulti-page menu. The selection_time out_pts field defines the validinteraction period for user interaction to activate the buttons.

The IF statement (if(stream_model==‘0_(b)’)) shown in the figure meansthat the above described “composition_time_out_pts” and “selection_timeout_pts” fields are optional information that are provided only when the“stream_model” field shows “Multiplexed” type. In the case of ICS ofwhich stream model is “Preloaded” type, neither of the“composition_time_out_pts” and “selection_time_out_pts” fields isprovided.

The user_time_out_duration field shows the timeout at which on-screenpages presented in response to a user operation are removed. When theAlways-On U/I is used, pages from the second page onward (referred to assub pages) are presented in response to user operations. Thus, after thetimeout defined by the user_time_out_duration, all subpages are removedfrom the screen leaving the first page on-screen. When the Pop-Up I/O isused, not only the sub pages but also every page of the Multi-page menuis presented in response to user operations. Thus, after theuser_time_out_duration, all on-screen pages are removed leaving no pageremain presented (No Menu Display).

Next, a description is given to the meaning of theselection_time_out_pts and composition_time_out_pts fields in an Epoch.

FIG. 11 is a timing chart illustrating the functions of theselection_time_out_pts and composition_time_out_pts fields in an ICSbelonging to a DS_(n) in the case where the DS_(n) is an Epoch Start DS(n=1). The timing chart shows that the composition_time_out_pts valuespecifies the end of Epoch (Epoch END). The interactive display is validfor a period from the PTS(DS_(n)[ICS]) to the Epoch END. This period isreferred to as the “Interactive Display Period”.

The selection_time_out_pts specifies a time preceding the Epoch End.User interaction may be made during a period from the PTS(DS_(n)[ICS])to the time specified by the selection_time_out_pts value. This periodis referred to as the “valid interaction period”. As above, theselection_time_out_pts field defines the end time of a period whichstarts from the first presentation of the interactive display and duringwhich the interactive display is available for user interaction. Thecomposition_time_out_pts field defines a period starting from the firstpresentation of the interactive display during which the interactivedisplay in an Epoch remains valid and thus presented. This concludes thedescription of the selection_time out_pts and composition_time_out_ptsfields. Next, a description is given to the Multi-page menu statetransition defined by the selection_time_out_pts,composition_time_out_pts, and user_time_out_duration fields.

FIG. 12A is a view showing the state transition of the Multi-page menubased on the selection_time_out_pts, composition_time_out_pts, anduser_time_out_duration fields. FIG. 12A is a view showing the statetransition of the Multi-Page menu when the Pop-Up U/I is used. TheMulti-page menu shown in the figure has the following three states: NoMenu Display, 1^(st) Page Only, Multi-Page Exist.

The 1^(st) page only is the state where only the first page of theMulti-Page menu is presented.

The Multi-Page exist is the state where pages from the second pageonward (i.e. sub pages) are presented in addition to the first page.

An arrow jt1 represents the transition from the No Menu Display state tothe 1^(st) Page Only state. This state transition is triggered by anevent “Pop-Up_On”. The “Pop-Up_On” is an event generated within theplayback apparatus in response to a user operation for a menu call.

An arrow jt2 represents the transition from the 1^(st) Page Only stateto the No Menu Display state. This state transition is triggered by anevent “Pop-Up_Off”. The “Pop-Up_Off” is an event generated within theplayback apparatus in response to a user operation for a menu removal.

Arrows jt3 and jt4 represent the transition from the 1^(st) Page Onlystate to the Multi-Page Exist state, and then from the Multi-Page Existstate to the 1^(st) Page Only state. The state transition is triggeredby an event “ChangeButtonPage” generated upon the timeout defined by theselection_time_out_pts. The ChangeButtonPage is an event generatedwithin the playback apparatus in response to a user operation forswitching a menu page. The event is generated on precondition that auser operation for switching a menu page is made. Theselection_time_out_pts shows the time for forcefully invalidating abutton for receiving a user operation for switching a menu page. Thus,the description of the selection_time_out_pts field realizes the statetransition of “1^(st) Page Only→Multi Page Exist→1^(st) Page Only”.

An arrow jt5 represents the transition from the 1^(st) Page Only stateto the No Menu Display state. An arrow jt6 represents the transitionfrom the Multi-Page Exist state to the No Menu Display state. The statetransition jt6 is triggered by the timeout of the user_timeout_duration.

In the figure, the state transitions to the No Menu Display state fromthe 1^(st) Page Only state (jt5) and from the Multi-Page Exist state aredefined by the user_time_out_duration field. That is to say, it ispossible to define such control that on-screen menu pages areautomatically removed when the AV playback reaches to a climax scene.With this control, it is avoided that the menu pages interferes withviewing the video stream playback.

FIG. 12B shows the state transition of Multi-page menu when theAlways-On U/I is used. An arrow at1 represents the transition from theNo Menu Display state to the 1^(st) Page Only state. This statetransition is triggered by an event “Epoch Start”. The “Epoch Start” isan event showing that the current playback point reaches the pointcorresponding to the PTS associated with the interactive_compositionstructure. It is shown that the 1^(st) page is automatically presentedas the video stream playback proceeds and without waiting for any useroperation.

An arrow at2 shows the transition from the 1^(st) Page Only state to theMulti-Page Exist state, and an arrow at3 shows the transition from theMulti-Page Exist state to the 1^(st) Page Only state. These statetransitions are also triggered by the ChangeButtonPage event generatedupon a timeout of the valid interaction period defined by theselection_time_out_pts.

An arrow at4 represents the transition from the Multi-Page Exist stateto the 1^(st) Page Only state. This state transition is triggered by thetimeout of the user_time_out_duration. As apparent also from this statetransition, when the Always-On U/I is used, the state transition is madeto the 1^(st) Page Only state not to the No Menu Display state upon thetime but of the user_time out_duration.

An arrow at5 represents the transition from the 1^(st) Page Only stateto the No Menu Display state. This state transition is triggered by thetime out of the composition_time_out_pts. Note that thecomposition_time_out_pts shows the end of Epoch (EpochEND).

As apparent from FIGS. 12A and 12B, the Multi-Page menu behavior isdefined by the selection_time_out_pts, user_time_out_duration, andcomposition_time_out_pts fields provided in the interactive_compositionstructure.

FIGS. 13-15 show the screen display presented under control defined asabove. FIG. 13 shows the display scenario incorporating the Multi-Pagemenu. In this scenario, the Multi-Page menu is not presented on screenat first. The 1^(st) page of the Multi-Page menu is presented when thecurrent playback point reaches the point t1, and a Sub page isadditionally presented at the point t2. When the current playback pointreaches the point t3, the on-screen Sub page is removed leaving only the1^(st) page on screen. The 1^(st) page is then removed at the point t4leaving no menu page displayed on screen.

At the bottom of FIG. 13 is the interactive_composition structuredescribed to achieve the above display scenario. The PTS(DS_(n)[ICS]) isset to the value corresponding to the point t1, theselection_time_out_pts to the value corresponding to the point t2, theuser_time_out_duration to the value corresponding to the point t3, andthe composition_time_out_pts to the value corresponding to the point t4.With this setting, the Multi-Page menu undergoes the state transitionsshown in FIG. 13.

Now, a description is given to the setting of selection_time_out_ptsvalue in greater detail with reference to a specific example.

FIGS. 14A-14C show a display example defined by theselection_time_out_pts. As shown in FIG. 14A, at the point of t1, thefirst page of Multi-Page Menu is overlaid with a picture of the videostream. A picture presented at the point t2 is of a scenic shot and isof less importance in the story of the AV clip. Since the picturepresents no valuable contents for users, the selection_time_out_ptsvalue is set to automatically activate a button presented on the 1^(st)Page at the point t2. Consequently, a Sub Page is overlaid with thescenic short shown in FIG. 14B, so that a composite image as shown inFIG. 14C appears on screen.

In a similar manner, a description is given to the setting ofuser_time_out_duration value with reference to a specific displayexample. FIGS. 15A-15D show a display example defined by theuser_time_out_duration. In this example, the Multi-Page menu is in theMulti-Page Exist state_where multiple Sub Pages (2^(nd), 3^(rd), and4^(th) pages) are presented on screen as shown in FIG. 15A. In theMulti-Page Exist state, the pages of the Multi-Page menu are overlaidwith a picture included in the video stream. Here, the picture to bepresented at the point t3 is a character image as shown in FIG. 15B. Inthis case, the multiple Sub Pages overlaid with the picture end upcovering most of the character image as shown in FIG. 15C. As a result,the facial expression of the character image can not be seen. To avoidthis interference, the user_time_out_duration is set to the valuecorresponding to a time immediately before the point t3. With thissetting, the Multi-Page menu undergoes the state transition from theMulti-Page Exist state to the 1^(st) Page Only state. As a result, thecomposite image as shown in FIG. 14D is presented at the point t3. Here,the 1^(st) page is solely presented on screen without blocking thecharacter image too much. Furthermore, since the 1^(st) page remains onscreen rather than removing every page of the Multi-Page Menu, there isno need to call the menu again.

As described above, according to the first embodiment, the ICSmultiplexed in the graphics stream includes control information definingbehavior of a Multi-Page Menu in accordance with the proceeding of AVplayback. This makes it possible to define such control that numerouspages are presented during the playback of less important scenes, suchas scenic shorts, and that on-screen pages are removed as the AVplayback approaches climax scenes. With such control, the menu pages arepresented without interfering with the playback of the AV clip, which isthe main content. Thus, the user can concentrate on viewing the AVplayback while enjoying convenience of the Multi-Page Menu functions.

Second Embodiment

A second embodiment relates to display compositions for presenting pagesof a Multi-Page Menu. According to the second embodiment, a displaycomposition is designed to present a plurality of graphical elementscalled button on screen. In addition, an animation effect is presentedon screen before introducing and/or after removing a menu page.

FIG. 16 is a view showing the internal structure of page informationassociated with an arbitrary one of multiple pages (x^(th) page) of theMulti-Page Menu. As shown in the left part of the figure, pageinformation (x) is composed of the following fields: “page_id” showingan identifier uniquely identifying the page(x); “UO_mask_table”;“in_effects” specifying an animation effect to be presented asintroduction of the page x to the screen; “out_effects” specifying ananimation effect to be presented after removal of the page(x) from thescreen; “animation_frame_rate_code” showing an animation frame rate tobe followed at the time of presenting animation effects for the page(x);“default_selected_button_id_ref”, “default_activated_button_id_ref”;“pallet_id_ref”; and “button_info (1), (2) . . . (number_of_buttons-1)”each associated with one of the buttons presented on the page(x).

The UO_mask_table field shows permissions and prohibitions of useroperations on the page (x). If this field is set to a value showing“prohibited”, a corresponding user operation to the playback apparatusis invalid while the page (x) is active.

The default_selected_button_id_ref field shows whether a button to beselected as default upon starting the presentation of page (x) isdetermined dynamically or statically. When this field is set to “0xFF”,the default selected button is determined dynamically. In such a case,the value held in a Player Status Register (PSR) of the playbackapparatus is interpreted with priority, and the button shown by the PSRis put into the selected state. On the other hand, when this field isset to the value other than “0xFF”, the default selected button isdetermined statically. In such a case, the PRS is overwritten with thedefault_selected_button_id_ref field value, and the button specified bythis field is put into the selected state.

The default_activated_button_id_ref field shows a button to beautomatically activated at the time defined by theselection_time_out_pts field value. When thedefault_activated_button_id_ref field value is set to “FF”, the buttonthat is in the selected state at the time of the timeout isautomatically activated. On the other hand, when this field is set to“00”, no button is automatically activated. When this field is set tothe value that is neither “00” or “FF”, the value is interpreted as avalid button number statically specifying the button to be automaticallyactivated.

The pallet_id_ref field shows the ID associated with a pallet to be setin a CLUT unit, which will be described later.

Each button_info structure is information defining a button to bepresented on the page (x). With the above fields, each page of theMulti-Page menu is defined. Next, a description is given to the internalstructure of button_info structure. In the description, an arbitrary oneof the buttons on the page (x) is denoted as a button(i). In FIG. 16,arrows cx1 indicate that the internal structure of button_info (i) isexcerpted to be shown in detail.

Each button presented on the page has three states which are a normalstate, a selected state, and an activated state. When a button is in thenormal state, the button is simply displayed. When a button is in theselected stated, the button is currently focused as a result of a useroperation but not yet activated. When a button is in the activatedstate, the button has been activated. Since each button has these threestates, the button_info(i) is provided with the following information.

The button_id field shows a value uniquely identifying the button(i) inthe interactive_composition structure.

The button_numeric_select_value field shows a flag indicating whetherthe button(i) is numerically selectable.

The auto_action_flag field shows whether the button(i) is automaticallyactivated. When the auto_action_flag field is set to ON (bit value of“1”), the button(i) is transferred not to the selected state butdirectly to the activated state when selected. On the other hand, whenthe auto_action_flag field is set to OFF (bit value of “0”), thebutton(i) is transferred not to the activated state but to the selectedstate when selected.

The button_horizontal_position field and button_vertical_position fieldrespectively specify the horizontal and vertical positions of the topleft pixel of the button(i) on the interactive display.

The neighbor_info structure shows buttons to receive the selected statein response to user operations made to move a focus in an up, down,right, and left direction when the button (i) is in the selected state.The neighbor_info structure is composed of upper_button_id_ref,lower_button_id_ref, left_button_id_ref, and right_button_id_ref fields.

The upper_button_id_ref field specifies the button number of a button toreceive the selected state if a user operation instructing to move thefocus in the up direction is made while the button (i) is in theselected state. The move up operation is executed at a push of a Move Upkey of the remote controller. If this field is set to the same buttonnumber as that of the button(i), the user operation to the Move Up keyis ignored.

Similarly, the lower_button_id_ref field, the left_button_id_ref field,the right_button_id_ref field each specify the button numbers of abutton to receive the selected state if a user operation instructing tomove the focus to the down, left, or right direction is made, while thebutton(i) is in the selected state. The respective operations areexecuted at a push of the Move Down key, Move Left key, and Move Rightkey of the remote controller. If these fields are set to the same buttonnumber as that of the button (i), the user operations to the those keysare ignored.

The normal_state_info structure defines the normal state of thebutton(i), and is composed of the normal_start_object_id_ref,normal_end_object_id_ref, and normal_repeat_flag fields.

The normal_start_object_id_ref field specifies the first one of theobject_id values that are serially assigned to a sequence of ODSs usedto present the normal state of button (i) in animation.

The normal_end_object_id_ref field specifies the last one of theobject_id values that are serially assigned to the sequence of ODSs usedto present the normal state of button(i) in animation. If thenormal_end_object_id_ref field specifies the same ID value as that ofthe normal_start_object_id_ref, the static image of a graphics objectidentified by that ID is presented as the button(i).

The normal_repeat_flag field specifies whether the animation of thebutton(i) in the normal state is to be continuously repeated.

The selected_state_info structure defines the selected state of thebutton(i). The selected_state_info structure is composed of theselected_state_sound_id_ref, selected_start_object_id_ref,selected_end_object_id_ref, and selected_repeat_flag fields.

The selected_state_sound_id_ref field specifies sound data to bereproduced as a click sound when the button(i) is put to the selectedstate. The sound data is specified by the sound_id of a piece of sounddata contained in the file called sound.bdmv. When this field is set to“0xFF”, no sound data is associated with the selected state of thebutton (i) and thus no click sound is reproduced when the button(i) ischanged to the selected state.

The selected_start_object_id_ref field specifies the first one of theobject_id values that are serially assigned to a sequence of ODSs usedto present the selected state of button(i) in animation.

The selcted_end_object_id_ref field specifies the last one of theobject_id values that are serially as signed to the sequence of ODSsused to present the normal state of button(i) in animation.

If the selected_end_object_id_ref field specifies the same ID value asthat of the selected_start_object_id_ref, the static image of a graphicsobject identified by the ID is presented as the button(i).

The selected_repeat flag field specifies whether the animation of thebutton(i) in the selected state is to be continuously repeated. If theselcted_start_object_id_ref and selcted_end_object_id_ref fields havethe same value, the selected_repeat_flag field is set to the value of“00”.

The activated_state_info structure defines the activated state of thebutton(i), and is composed of the activated_state_sound_id_ref,activated_start_object_id_ref, and activated_end_object_id_ref fields.

The activated_state_sound_id_ref field specifies sound data to bereproduced as a click sound when the button(i) is put to the activatedstate. The sound data is specified by the sound_id of a piece of sounddata contained in the sound.bdmv file. When this field is set to “0xFF”,no sound data is associated with the activated state of the button(i)and thus no click sound is reproduced when the button(i) is changed tothe activated state.

The activated_start_object_id_ref field specifies the first one of theobject_id values that are serially assigned to a sequence of ODSs usedto present the activated state of button(i) in animation.

The activated end_object_id_ref field specifies the last one of theobject_id values that are serially assigned to the sequence of ODSs usedto present the activated state of button (i) in animation.

The “navigation_command” structure shows a navigation command to beexecuted when the button(i) is activated. A representative example ofthe navigation command is, a SetButtonPage command. The SetButtonPagecommand instructs the playback apparatus to display a desired page ofthe Multi-Page menu with a desired one of buttons on the page in theselected state. With the use of such navigation commands, contentcreators can readily describe page transitions at the time of authoring.

This concludes the description of the button_info structure. Now, areference is made to FIG. 18 showing an example of the button_infodescription defining the state transition of buttons O-A to O-D, asillustrated in FIG. 17. In FIG. 17, arrows hh1 and hh2 represent thestate transitions defined by the neighbor_info ( ) structure of thebutton info(1), which is associated with the button O-A. In the buttoninfo(1), the lower_button_id_ref field is set to the value specifyingthe button O-C. Thus, if a user operation is made to the Move Down key(FIG. 17, up1) while the button O-A is in the selected state, the buttonO-C receives the selected state (sj1). The right_button_id_ref field inthe button info(1) is set to the value specifying the button O-B. Thus,if a user operation is made to the Move Right key (up2) while the buttonO-A is in the selected state, the button O-B receives the selected state(sj2).

An arrow hh3 in FIG. 17 represents the button state transition definedby the neighbor_info( ) structure in the button_info(2), which isassociated with the button O-C. In the button_info(2), theupper_button_id_ref field is set to the value specifying the button O-A.Thus, if a user operation is made to the Move Up key (up3) while thebutton O-C is in the selected state, the button O-A is put back into theselected state (sj3).

Now, a description is given to the graphical images of the buttons O-A,O-B, O-C, and O-D. A DS_(n) provided with the ICS shown in FIG. 18includes ODSs 11-19 corresponding to the graphical images shown in FIG.19. The normal_start_object_id_ref and normal_end_object_id_ref fieldsin the button_info(0) specify the ODSs 11 and 13, respectively. Thus,the normal state of the button O-A is presented in animation using thesequence of the ODSs 11-13. Similarly, the selected_start_object_id_refand selected_end_object_id_ref fields in the button_info(0) specify theODSs 14 and 16, respectively. Thus, the selected state of the button O-Ais presented in animation using the sequence of the ODSs 14-16. Withthis setting, when a user operation is made to put the button O-A to theselected state, the graphical image serving as the button O-A is changedfrom the one presented using the ODSs 11-13 to the one presented usingthe ODSs 14-16. Here, if the normal_repeat_flag field in thenormal_state_info( ) structure is set to the value “1”, the animatedpresentation of the ODSs 11-13 is repeated as indicated by “→(A)” and“→(A)” in the figure. Similarly, if the selected_repeat_flag field inthe selected_state_info( ) structure is set to the value “1”, theanimated presentation of the ODSs 14-16 is repeated as indicated by“→(B)” and “→(B)” in the figure.

As above, a plurality of sets of ODSs that can be presented in animationare associated with the buttons O-A, O-B, O-C, and O-D. Thus, with theICS referencing the ODSs, such a control is defined that a characterimage serving as a button changes its facial expression in response touser operations.

This concludes the description of the button_info structure. Next, adescription is given to animation effects. As shown in FIG. 16, thein-effects field and the out-effects field each specify aneffect_sequence structure. FIG. 20 is a view showing an effect_sequencestructure. As shown in the left half of the figure, the effect_sequencestructure is composed of: window_info(0), (1), (2) . . .(number_of_windows-1); and the effect_info(0), (1), (2) . . .(number_of_effects-1).

The animation effect is presented by updating display compositions atfixed intervals. Each effect_info structure is a piece of informationdefining one display composition.

Arrows ec1 indicate that the internal structure of effect_info isexcerpted to be shown in detail. As shown in the figure, the effect_infostructure is composed of: the effect_duration field showing theabove-mentioned fixed interval, i.e. a time period for which theassociated display composition is to be presented; the pallet_id_reffield specifying a pallet used for the associated display composition;and the composition_object (0), (1), (2) . . .(number_of_composition_object-1) specifying the details of the displaycomposition.

Each window_info structure defines a window or an area on the graphicsplane within which the display compositions are rendered. As indicatedby arrows wc1, the window_info is excerpted to be shown in detail. Asshown in the figure, the window_info is composed of the followingfields: “window_id” uniquely identifying a widow on the graphics plane;“window_horizontal_position” specifying the horizontal position of thetop left pixel of the window; “window_vertical_position” specifying thevertical position of the top left pixel of the window; “window_width”specifying the width of the window; and “window_height” specifying theheight of the window.

Now, a description is given to values that may be set in each of thewindow_horizontal_position, window_vertical_position, window_width, andwindow_height fields. These fields are set to the value corresponding tothe coordinates within the two-dimensional graphics plane having theheight and width defined by the video_height and video_width fields.

Since the window_horizontal_position field shows the horizontal addressof the pixel on the graphics plane, this field takes on a value from “1”to the video_width value. Similarly, since the window_vertical_positionshows the vertical address of the pixel on the graphics plane, thisfield takes on a value from “1” to the video_height value.

Since the window_width field shows the width of a window on the graphicsplane, this field takes on a value from “1” to the value calculated bysubtracting the window_horizontal_position value from the video_widthvalue. Similarly, since the window_height field shows the height ofwindow on the graphics plane, this field takes on a value from “1” tothe value calculated by subtracting the window_vertical_position valuefrom the video height value.

As above, the window_info is provided with window_horizontal_position,window_vertical_position, window_width, window_height fields definingthe size and postion of a window on the graphics plane.

Now, a description is given to the constraint on the window size forrealizing the framework for presenting display compositions of graphicsat a rate of 24 frames per second. This frame rate corresponds to thevideo frame rate applied to the video stream playback. With thisframework, the graphics objects are presented in detailedsynchronization with the video stream. To realize the framework, thewindow must be cleared and rendered at the video frame rate. Here, aconsideration is given to the required transfer rate from the objectbuffer to the graphics plane.

First, a constraint on the window size is examined. Let Rc denote thetransfer rate from the object buffer to the graphics plane. In theworst-case scenario, the clearing and rendering of the window need to beperformed at Rc. In other words, the clearing and rendering need to beperformed at half of Rc (Rc/2).

Thus, the following equation needs to be satisfied.

(Window Size)×(Frame Rate)≈Rc/2

When the frame rate is 24, then RC is expressed in the followingequation.

Rc=(Window Size)×2×24

When the total number of pixels of the graphics plane is 1920×1080 andthe bit length of an index per pixel is 8 bits, the total capacity ofthe graphics plane is 2 Mbytes (≈1920×1080×8).

When Rc is 128 Mbps and the window size is 1/A of the graphics plane,the following equation is satisfied.

128,000,000=1920×1080×1/A×8×2×24

1 /A≈0.16(=128,000,000/(1920×1080×8×2×24))

Thus, by restricting the window size to 16% of the entire graphicsplane, the synchronization between animation effects with the videostream is guaranteed as long as the animation effects are rendered atthe transfer rate of 128 Mbps.

Suppose, for example, the effect_duration showing the rate of clearingand rendering a window is ½ or ¼ of the video frame rate. In this case,the window size can be as large as 32% or 64% of the entire graphicsplane even if Rc is the same. That is to say, the effect_sequencestructure has a feature that the effect_duration is determined to belonger for a larger window size, and shorter for a smaller window size.This concludes the description of the window size. Next, a descriptionis given to the position of window. As mentioned above, the position andthe size of a window are fixed throughout an Epoch.

The position and the size of a window are fixed throughout the durationof an Epoch for the following reason. When the position or size of awindow is changed, the write address of the graphics plane needs to bechanged, which incurs overhead. The overhead inevitably reduces thetransfer rate from the object buffer to the graphics plane.

There is a constraint on the number of graphics objects allowed to berendered in one window at a time. The number of graphics objects isrestricted in order to reduce overhead incurred at the time oftransferring decoded graphics objects. To be more specific, the overheadreferred herein is incurred at the time of setting addresses of edgeparts of the graphics objects. This overhead increases if the number ofedge parts is greater.

If there is no restriction on the number of graphics objects that can bedisplayed in one window, the overhead is incurred unlimitedly whentransferring graphics objects to the graphics plane, which increasesfluctuations in transfer load. On the other hand, if the number ofgraphics objects in one window is limited to two, the transfer rate Rccan be set on an assumption that the number of instances of overhead isat most four. Hence a minimum standard for the transfer rate Rc can bedetermined easily.

Turning now to the effect_info structure, the effect_duration isdetermined depending on the window size. Now, a description is given tohow the effect_duration is determined.

FIG. 21A is a view schematically illustrating how the effect_duration isdetermined in the figure, the (j+1)^(th) display composition ispresented after a plurality of video frames from the presentation of thej^(th) display composition The number of the plurality of video framesis the effect_duration. In order to calculate the effect_duration, thetotal size of data to be rendered for presentation of each displaycomposition must be known. Here, the effect_sequence is provided withwindows of the number that is indicated by the number_of_windows fieldvalue. Thus, the total size of data needed to be rendered forpresentation of the j^(th) display composition is equal to the totalsize of the windows (0) through (number_of_windows-1), as indicated bythe brace br.

As above, the j^(th) display composition is provided with the windows(0) through (number_of_windows-1). Thus, the total data size of thewindows is divided by the transfer rate of the playback apparatus (128Mbps) and further multiplied by the time accuracy of PTS (90 KHz), sothat the result is expressed in 90 KHz. The resulting value is theduration for which the j^(th) display composition of the effect_sequenceis presented. FIG. 21B shows the equation for calculating theeffect_duration.

Note that in this specification, the operator “ceil( )” represents adivide that returns a value rounded up to an integer.

FIG. 22 is a view showing a specific example of an in-effect animation.The in-effect animation presents an image such that a flash of lightappears on the palm of hand of the character image on screen and a menugradually rises up after the flash of light disappears. The size ofgraphics data is the largest within the in-effect at the time when themenu is fully shown on screen (effect (4)). Thus, the window_width andwindow_height of a window are determined based on the data size of theeffect (4). Suppose, the window size determined herein is about 50% ofthe entire graphics plane. This window size is nearly three times aslarge as the above-mentioned window size of 16% of the graphics plane.Consequently, the effect_duration for this in-effect is determined to belonger in order to allow the display composition to be duly updated atthe rate of 128 Mbps. Each effect_duration of the effects (0), (1), (2),and (3) is set to be relatively long. According to this setting, eachdisplay composition is rendered for every three video frames, so that itis ensured that the in-effect animation is seamlessly presented. Thesame scheme applies to an out-effect animation. FIG. 23 is a viewshowing the out-effect animation for which window_width, window_height,and effect_duration field values are determined in the manner describedabove.

As described above, according to the second embodiment, the window_infodefines an area in the graphics plane for rendering, and the effect_infodefines a duration at intervals of which the rendering area is updated.Thus, the duration may be adjusted to be longer when the rendering areais set to be larger, and shorter when the rendering area is set to besmaller. With this setting, the seamless presentation of animationeffects is ensured as long as the playback apparatus transfers graphicsdata at a fixed transfer-rate.

Third Embodiment

A third embodiment of the present invention relates to an improvement onthe presentation of animation effects, such as Scroll, Wipe-In/Out, andCut-In/Out. The structure for achieving the animation effects lies inthe composition_object structures shown in FIG. 20. FIG. 24 is a viewshowing the internal structure of an arbitrary composition_object(i) indetail. As shown in the figure, the composition_object(i) is composed ofthe following fields: “object_id_ref”; “window_id_ref”,“object_cropped_flag”; “composition_object_horizontal_position”;“composition_object vertical_position”, and “cropping_rectangle_info(1),(2) . . . (n)”.

The object_id_ref field shows a reference value for a graphics objectidentifier (object_id). The reference value corresponds to the object_ididentifying the graphics object that is used for generating a displaycomposition according to the composition_object(i).

The window_id_ref field shows a reference value for a window identifier(window_id). The reference value corresponds to the window_ididentifying the window in which the display composition according to thecomposition_object(i) is rendered.

The object_cropped flag field shows a flag indicating whether thegraphics object cropped on the object buffer is to be presented or notto be presented. When this field is set to “1”, the graphics objectcropped on the object buffer is presented. On the other hand, when thisfield is set to “0”, the cropped graphics object is not presented.

The composition_object_horizontal_position field shows the horizontalpostion of the top left pixel of the graphics object on the graphicsplane.

The composition_object_vertical_position field shows the verticalposition of the top left pixel of the graphics object on the graphicsplane.

The cropping_rectangle structure is valid when the object_cropped_flagfield is set to “1”. In FIG. 24, arrows wd2 indicate that the internalstructure of cropping_rectangle is excerpted to be shown in detail. Asshown in the figure, the cropping_rectangle structure is composed of thefollowing fields: “object_cropping_horizontal_position”;“object_cropping_vertical_position”; “object_cropping_width”; and“object_cropping_height”.

The object_cropping_horizontal_position field shows the horizontalposition of the top left corner of the cropping rectangle. The croppingrectangle defines the region, of the graphics object to be cropped.

The object_cropping_vertical_postion field shows the vertical postion ofthe top left corner of the cropping rectangle.

The object_cropping_width field shows the width of the croppingrectangle.

The object_cropping_height field shows the height of the croppingrectangle.

This concludes the description of the structure of composition_object.Next, a description is given to a specific example of thecomposition_object structure.

FIG. 25 is a view showing a specific example of an in-effect presentinga Right-Scroll animation effect. The in-effect animation of this examplepresents an image such that a language selection menu gradually appearsfrom the right edge of the screen. For this in-effect animation,composition_object(0), composition_object(1), composition_object(2), andcomposition_object(3) are allocated to points t0, t1, t2, and t3,respectively. Furthermore, the ICS and the effect_info structureassociated with each of the DSs are defined to include the followingwindow_info structures and composition_object structures.

Now, a description is given to the setting of each composition_objectstructure. FIGS. 26-29 each illustrate an example setting of thecomposition_object structure. FIG. 26 is a view showing an example ofthe composition_object(0) structure.

In the figure, the window_horizontal_position andwindow_vertical_position fields specify the coordinates of the top leftpixel LPO of a window on the graphics plane. The window_width andwindow_height fields specify the width and height of the window.

As shown in the figure, the object_cropping_horizontal_position andobject_cropping vertical_position fields specify a reference point ST1used for determining a region of the graphics object to be cropped. Thereference point ST1 is addressed based on the coordinate system whichhas an origin at the top left corner of the graphics object on theobject buffer. The region having the length and width specified by theobject_cropping_width and object_cropping_height fields from thereference point ST1 is determined as the cropping rectangle (solid-linebox in the figure). The cropped part of the graphics object is renderedon the graphics plane at the position having the top left pixel LPOspecified by the composition_object_horizontal_position andcomposition_object_vertical_position fields. In this example, about ¼ ofthe language selection menu from the right is rendered within the windowon the graphics plane. As a result, the right one-quarter of thelanguage selection menu is presented as a composite image overlaid onthe video.

FIGS. 27, 28, and 29 are views showing examples of thecomposition_object structure(1), composition_object structure(2), andcomposition_object structure(3), respectively. The window_info structureshown in each of the figures is identical with the one shown in FIG. 26.Thus, the description is omitted. The composition_object (1) structureshown in FIG. 27, however, differs from the one shown in FIG. 26. InFIG. 27, the object_cropping_horizontal_position andobject_cropping_vertical_position fields specify the coordinates of thetop left pixel of about a right half, rather than ¼, of the languageselection menu stored on the object buffer. The object_cropping_heightand object_cropping_width fields specify the height and width of theright half of the language selection menu. Similarly, in FIG. 28, therespective fields of the composition_object (2) structure togetherspecify about ¾ of the language selection menu from the right. In FIG.29, the respective fields of the composition_object(3) structuretogether specify the entire part of the language selection menu. Byrendering the cropped part of the graphics object as shown in FIGS. 27,28, and 29 to the graphics plane, the right half of the languageselection menu, the ¾ of the language selection menu from the right, andthe entire language selection menu are overlaid on the video.

As described above, according to the third embodiment of the presentinvention, with the composition_object structures, an animation effectis easily defined as desired. For example, it is easily defined that agraphics object stored on the object buffer is rendered so as to make agradual appearance on screen or gradual disappearance from the screen.

Fourth Embodiment

A fourth embodiment of the present invention relates to an improvementmade through optimizing the order of ODSs within a DS. As describedabove, the ODSs within a DS are referenced by theinteractive_composition so as to present an in-effect or out-effectanimation or one state of a button. The order of ODSs within a DS isdetermined depending on whether the ODSs are used for an in-effectanimation, an out effect animation, or a button state.

Specifically, the ODSs within a DS are grouped into (1) ODSs used forpresentation of an in-effect animation for Page[0], (2) ODSs used forpresentation of the normal state of a button, (3) ODSs used forpresentation of the selected state of a button, (4) ODSs used forpresentation of the activated state of a button, (5) ODSs used forpresentation of an in-effect for Page[0] and an in-effect and out-effectanimations for Page[1] onward. That is to say, ODSs associated with thesame presentation is put into the same group. The group of ODSsassociated with the in-effect is referred to as an “in-effect group”.The group of ODSs associated with each button state is referred to as a“button-state group”. The groups of ODSs associated with the out-effectof Page[0], in-effects and out-effects of Page[1] onward are referred toas a “remain group”.

Those groups of ODSs are arranged in the following order: in-effectgroup—normal-state group→seleted-state group—activated-stategroup→remain group. In this way, the order of ODSs within a DS isdetermined based on the presentation targets with which the ODSs areassociated.

Next, a description is given specifically to the order of ODSs within aDS_(n). FIG. 30 is a view showing the order of ODSs within the DS_(n).

In the figure, ODS₁-ODS_(g) belong to a group of ODSs associated withthe in-effect for Page[0] (in-effect group).

ODS_(g+1)-ODS_(h) are a group of ODSs associated with the normal stateof buttons (ODSs for Normal state).

ODS_(h+1)-ODS_(i) belong to a group of ODSs associated with the selectedstate of the buttons (ODSs for Selected state).

ODS_(i+1)-ODS_(j) belong to a group of ODSs associated with theactivated state of the buttons (ODSs for Activated state).

ODS_(j+1)-ODS_(k) belong to a group of ODSs associated with theout-effect of Page[0] and ODSs associated with the in-effects andout-effects of pages from Page[1] onward (remain group).

The groups of ODSs are arranged in the following order: in-effectgroup→normal-state group→selected-state group→activated-stategroup→remain group. This order is determined so that ODSs needed for thefirst presentation of interactive display are read first, whereas ODSsneeded only after a display update are read later.

Next, a description is given to the order of ODSs when there is amultiple-reference. The term “multiple-reference” used herein means thatthe same object_id is referenced by two or more pieces ofnormal_state_info, selected_state_info, and activated_state info withinan ICS. For example, with the multiple references, a specific graphicsobject used to render a button in the normal state is commonly used torender another button in the selected state. In other words, thegraphics object is shared, so that the number of ODSs can be reduced.Here, a problem arises as to which button-state group an ODS having themultiple-reference belongs.

Specifically, when one ODS is associated with one button in the normalstate as well as another button in the selected state, it needs to bedetermined whether the ODS belongs to the button-state group for thenormal state or the button-state group for the selected state.

In such a case, the ODS placed in the button-state group that firstappears in the ICS.

For example, if an ODS is referenced by both the normal-state andselected-state groups, the ODS is placed in the normal-state group(N-ODSs) but not in the selected-state group (S-ODSs). Also, if an ODSis referenced by both the selected-state and activated-state groups, theODS is placed in the selected-state group (S-ODSs) but not in theactivated-state group (A-ODSs). In this manner, each ODS referenced bytwo or more different state_info structures, the ODS appears only oncein the DS and placed in the first one of the button-state groups.

The same applies to the ODSs associated with animation effects. If agraphics object is commonly used in the in-effect and the out-effect forPage[0], the graphics object appears only once in the DS and placed inthe in-effect group. Similarly, if a graphics object is commonly used inthe in-effect for Page[0] and the in-effect or out-effect for Page[1]onward, the graphics object appears only once in the DS and placed inthe in-effect group. This concludes the description of the order of ODSswhere there is a multiple-reference to an ODS.

Because of possible multiple-references to ODSs as described above, theDS_(n) may not contain all the ODS groups mentioned above. For example,the DS_(n) may not contain the selected-state group ODSs due to amultiple-reference. Furthermore, the DS_(n) may not contain thein-effect group ODSs. This is also because an in-effect is not essentialto the presentation of interactive display.

Now, a description is given to the order of ODSs within the button-stategroup for the selected state (S-ODSs). Among the S-ODSs, which ODSsshould be placed at the beginning depends on whether a default selectedbutton is determined statically or dynamically. The default selectedbutton is statically determined if the default_selected_button_id_reffield in the interactive_composition is set to a valid value other than“00”. This valid value specifies the button to be selected as default.In the case where the default_selected_button_id_ref field value isvalid and that ODSs associated with the specified default selectedbutton are not present in the N-ODSs, the ODSs are placed at thebeginning of S-ODSs.

On the other hand, if the default_selected_button_id_ref field is set to“00”, a different button is dynamically selected as default depending onthe state of playback apparatus.

The default_selected button_id_ref field is set to the value “00” in thecase, for example, where an AV Clip into which the DS is multiplexedserves as a merge point of a plurality of playback paths. In such acase, a different button needs to be selected as default depending onwhich playback path has been taken. Thus, it is not necessary to placespecific ODSs at the beginning of S-ODSs.

FIG. 31 is a view showing how the order of ODSs within S-ODSs differsbetween the cases where the default_selected_button_id_ref field is setto “00” and to the valid value specifying the button B. In the figure,arrows ss1 indicate the order of ODSs within S-ODSs in the case wherethe default_selected_button_id_ref field specifies the button_ididentifying the button B. Arrows ss2 indicate the order of ODSs within8-ODSs in the case where the default_selected_button_id_ref field is setto “00”. As shown in the figure, in the case where thedefault_selected_button_id_ref field specifies the button B, the ODSsassociated with the selected state of button B (ODSs for Button B) areplaced at the beginning of the S-ODSs, followed by the ODSs associatedwith the other buttons (ODSs for Buttons A, C, and D). On the otherhand, in the case where the default_selected_button_id_ref field is setto “00”, the ODSs associated with the selected state of button A areplaced at the beginning of the S-ODSs. As described above, the order ofODSs within S-ODSs differs depending on whether thedefault_selected_button_id_ref field value is valued or not.

According to the fourth embodiment, the ODSs within an DS are arrangedin the order of appearance in the DS. Thus, the display compositioncontaining a number of graphics elements can be smoothly presented.

Fifth Embodiment

In the first embodiment, how to allocate DSs to the time axis of AV Clipplayback is described. In a fifth embodiment of the present invention, adescription is given to how to set PTS and DTS values.

First of all, a description is given to a mechanism of effectingsynchronous control based on the DTS and PTS within an ODS.

The DTS shows a time, with the time accuracy of 90 KHz, at whichdecoding of the associated ODS needs to be started. The PTS shows adeadline for completing the decoding.

The decoding of ODS can not be completed instantly, and takes a certainduration. In order to explicitly show the start and end times ofdecoding the ODS, the DTS and PTS show the decoding start time anddecoding deadline.

The PTS value shows the deadline by which decoding of the associated ODSneeds to be completed and the resulting uncompressed graphics objectneeds to be available on the object buffer of the playback apparatus.

The decoding start time of an arbitrary ODS_(j) belonging to a DS_(n) isshown by DTS (DS_(n)[ODS]) with the time accuracy of 90 KHz. Thus, thedecoding deadline of the ODS_(j) is determined by a sum of the DTS(DS_(n)[ODS]) value and a maximum time which may be taken for thedecoding.

Here, let SIZE (ODS_(n)[ODS_(j)]) denotes the size of the ODS_(j) afterdecompression, and Rd denotes the decoding rate for ODS_(j), then themaximum time taken for the decoding is ceil(SIZE (DS_(n)[ODS_(j)])).

By converting this maximum time to the accuracy of 90 KHz and adding theresult to the DTS of the ODS_(j), the decoding deadline to be shown bythe PTS is calculated with the accuracy of 90 KHz.

The PTS of the ODS_(j) belonging to the DS_(n) can be expressed by thefollowing equation.

PTS(DS[ODS_(j)])=DTS(DS_(n)[ODS_(j)])+90,000×ceil(SIZE(DS_(n)[ODS_(j)])/Rd)

In addition, the relation between two adjacent ODSs (ODS_(j) andODS_(j+1)) needs to satisfy the following equation.

PTS(DS_(n)[ODS_(j)])≦DTS(DS_(n)[ODS_(j+1)])

This concludes the description of the PTS and the DTS of an ODS. Next, adescription is given to the PTS within an ICS. The ICS needs to beloaded to the composition buffer of the playback apparatus before thedecoding start time of the first ODS within the DS_(n) (i.e. before thetime shown by DTS(DS_(n)[ODS₁])) and also before the first presentationof the DS, becomes valid (i.e. the time shown by PTS (DS_(n)[PDS₁])).Thus, the following equations need to be satisfied.

DTS(DS_(n)[ICS])≦DTS(DS_(n)[ODS₁])

DTS(DS_(n)[ICS])≦PTS(DS_(n)[PDS₁])

Next, a description is given to the PTS within the ICS of the DS_(n).The PTS (DS_(n)[ICS]) value is set to satisfy the following equation.

PTS(DS_(n)[ICS])≧PTS(DS_(n)[ICS])+DECODEDURATION+TRANSFERDURATION

The DECODEDURATION shows the time taken for decoding graphics objectsneeded for the first presentation of the DS_(n)(ICS). The decoding starttime is equal to the DTS (ICS) value at the earliest. Here, letLASTODSPTS denote the PTS associated with the graphics object of whichdecoding is completed at the latest among graphics objects needed forthe first presentation of DS_(n)(ICS), then the DECODEDURATION is equalto LASTODSPTS(DS_(n))−DTS(DS_(n)[ICS]).

The LASTODSPTS value takes on a different value depending on thedefault_selected_button_id_ref setting and the order of ODSs within theDS. FIGS. 32A, 32B, 33A, 33B, and 33C show different ODS orders toillustrate LASTODSPTS setting using the DTS of an ODS.

FIG. 32A shows the case where the default selected button is staticallydetermined, i.e. where the default_selected_button_id_ref field is setto a non-zero value. In this case, the LASTOBSPTS takes on the PTS valueof the first ODS within the S-ODSs (i.e. the value ofPTS(DS_(n)[ODS_(h+1)])). In the figure, the ODS_(h+1) is enclosed withinthe solid line box.

FIG. 32B shows the case where the default selected button is determineddynamically, i.e. where the default_selected_button_id_ref is set to“00”. In this case, the LASTODSPTS takes on the PTS value of the lastODS among the S-ODSs (i.e. the value of PTS (DS_(n)[ODS_(i)])). In thefigure, the ODS_(i) is enclosed within the solid line box.

FIG. 33A shows a DS that includes no ODS associated with the selectedstate. In this case, the LASTODSPTS takes on the PTS value of the lastODS among the N-ODSs (i.e. the value of PTS (DS_(n)[ODS_(h)])). In thefigure, the ODS_(h) is enclosed within the solid line box.

FIG. 33B shows a DS that includes no ODS associated with the normalstate. In this case, the LASTODSPTS takes on the PTS value of the lastODS among the in-effect ODSs (i.e. the value of PTS(DS_(n)[ODS_(g)])) Inthe figure, the ODS_(g) is enclosed within the solid line box.

FIG. 33C shows a DS that includes no ODS at all. In this case, theLASTODSPTS takes on the DTS value specified in the ICS of the DS_(n)(i.e. the value of DTS(DS_(n)[ICS])). In the figure, the ICS is enclosedwithin the solid line box.

Note that the above description applies to the case where the DS_(n) isnot an Epoch Start DS. If the DS_(n) is an Epoch Start DS_(n) adifferent description is applied.

In the case of an Epoch Start DS, the graphics plane may be clearedentirely. The time taken for clearing the graphics plane may be longerthan the time calculated by LASTODSPTS (DS_(n))−DTS(DS_(n)[ICS]).According to a target decoder model for the BD-ROM, the graphicsdecoding and the graphics plane clearing are performed by differentprocessing units (the graphics processor performs the decoding). Thus,the graphics decoding and the plane clearing are performed in parallel.Even if the decoding is completed for graphics data necessary forpresentation of the first display composition, the graphics date can notbe rendered to the graphics plane unless the plane has been cleared.Thus, the DECODEDURATION needs to set to the value equal to the planeclear duration. Suppose the graphics plane has the width and heightequal to the video_width and video_height field values, respectively.Furthermore, the transfer rate between the object buffer and thegraphics plane is 128 Mbps. In order to clear the entire graphics plane,the playback apparatus is required to overwrite the graphics planehaving the size of 8×video_width×video_height at the transfer rate of128 Mbps. Expressing the duration taken for this overwriting in 90 KHz,the time taken for clearing the graphics plane (referred to as“PLANECLEARTIME”) is calculated in the following equation.

PLANECLEARTIME=ceil((90000×8×DS_(n)[ICS].video_width×DS_(n)[ICS].video_height)/128,000,000)

FIG. 34A shows an algorithm for obtaining the DECODEDURATION value fromthe value calculated by LASTODSPTS(DS_(n))−DTS(DS_(n)[ICS]) and thevalue of PLANECLERATIME. In the algorithm, the larger one of theLASTODSPTS(DS_(n))−DTS(DS_(n)[ICS]) value and the PLANECLERATIME valueis determined as the DECODEDURATION value.

As shown in the figure, if the composition_state field of theDS_(n)(ICS) indicates that the DS_(n) is not an epoch start DS(if(DS_(n)(ICS) composition_state==EPOCH_START) else), the value ofLASTODSPTS(DS_(n))−DTS(DS_(n)[ICS]) is determined as the DECODEDURATIONvalue (return (LASTODSPTS (DS_(n))−DTS (DS_(n)[ICS]))). On the otherhand, if the composition_state field of the DS_(n)(ICS) indicates thatthe DS_(n) is an epoch start DS (if (DS_(n)(ICS)composition_state==EPOCHSTART)), the larger one of theLASTODSPTS(DS_(n))−DTS(DS_(n)[ICS]) value and the PLANECLERATIME valueis determined as the DECODEDURATION value (return(max(LASTODSPTS(DS_(n))−DTS(DS_(n)[ICS])))).

FIGS. 34B and 34C are views showing examples of DECODEDURATION settingdepending on whether or not the PLANECLEARTIME value is larger than theLASTODSPTS(DS_(n))−DTS(DS_(n)[ICS]) value. If the PLANECLEARTIME valueis larger than the LASTODSPTS(DS_(n))−DTS(DS_(n)[ICS]) value, thePLANECLEARTIME is determined as the DECODEDURATION value, as shown inFIG. 34B. On the other hand, if the PLANECLEARTIME value is smaller thanthe LASTODSPTS(DS_(n))−DTS(DS_(n)[ICS]) value, theLASTODSPTS(DS_(n))−DTS(DS_(n)[ICS]) value is determined as theDECODEDURATION value, as shown in FIG. 34C.

The TRANSFERDURATION shows the time taken for transferring graphics dataneeded for presentation of the first display composition from the objectbuffer to the graphics plane. The first display composition within a DSdiffers depending on whether an in-effect is provided for the DS andwhether the default selected button is determined statically ordynamically. Thus, the TRANSFERDURATION also differs depending on thefirst display composition. If the first display composition isassociated with an in-effect, the time taken for transferring theuncompressed graphics needed for the presentation of the in-effect isdetermined as the TRANSFERDURATION.

Here, the in-effect defined by the DS_(n)[ICS] is for introducingPage[0] and rendered in windows of the number that is indicated by thenumber_of_window field value. Thus, the total size of graphics dataneeded for the first display composition of the in-effect is equal tothe total size of the windows (0) through (number_of_windows-1). Thus,to calculate the time taken for transferring graphics data needed forpresentation of the first display composition within the in-effectsequence (EFFECTTD), the total window size is divided by the transferrate of the playback apparatus (128 Mbps). To express the result in thetime accuracy of the PTS (90 KHz), the result is further multiplexed by90 KHz. The equation for calculating the EFFECTTD (DS_(n)) is shown inFIG. 35A.

On the other hand, if the DS_(n) is not provided with any in-effect, thefirst display composition for presentation of the DS_(n)[ICS] is Page[0]of the Multi-Page menu. If the default selected button is staticallyspecified, Page[0] of the Multi-Page menu is presented as soon asuncompressed data needed for the selected state of the default selectedbutton and the normal state of the other buttons is transferred from theobject buffer to the graphics plane.

Here, Page[0] of the Multi-Page menu associated with the DS_(n)(ICS)includes the buttons of the number indicated by the number_of buttonfield value. Thus, the data size needed for presentation of the firstdisplay composition is calculated as follows. First, the total size iscalculated for the graphics data associated with the normal state of allthe buttons on the page, which are specified by DS_(n)(ICS).Page[0].Buttons from (0) through (number_of_Button-1). Then, to the thuscalculated total size, the size of graphics data associated with theselected state of the default selected button of Page[0] (SBSIZE(DS_(n), DS_(n)[ICS].PAGE[0].default_selected_button_id_ref)) is added.Further, from the resulting value, the size of graphics data associatedwith the normal state of the default selected button (NBSIZE(DS_(n),DS_(n)[ICS] PAGE [0] default_selected-button-Id-ref)) is subtracted toobtain the data size needed for presentation of the first displaycomposition.

The thus calculated data size needed for presentation of the firstdisplay composition is divided by the transfer rate of the playbackapparatus (128 Mbps), and multiplied by 90 KHz for expressing the resultin the time accuracy of PTS (90 KHz). Then, the transfer duration takenfor presentation of the first page is calculated. FIG. 35B shows theequation for calculating the PAGDEFAULTTD(DS_(n)) in the case where noin-effect is provided and the default selected button is staticallydetermined.

In the case where the default selected button is dynamically determined,the equation shown in FIG. 35B is not applicable because it is not knownwhich of the buttons on Page[0] is selected as default. Thus, theTRANSFERDURATION needs to be calculated for the worst case. The worstcase is where the button with the largest size is selected as defaultamong the buttons defined by the DS_(n)(ICS).Page[0]. Among the buttonsof DS_(n)(ICS).Page[0], the button with the largest size is denoted asLRG{button: buttonεDS_(n)[ICS].PAGE[0]. Button}.

Then, the data size needed to be transfer in the above-mentioned worstcase (the maximum data size) is calculated as follows. First, the totalsize of all the buttons on Page[0] (Page[0].Buttons from (0) through(number_of_Button-1)) is calculated. To the thus calculated total size,the size of the button LRG (BSIZE (DS_(n), LRG{button:buttonεDS_(n)[ICS].PAGE[0].Button})) is added. Further, from theresulting value, the size of graphics data associated with the normalstate of the button LRG (NBSIZE (DS_(n), LRG{button: buttonεDS_(n)[ICS]PAGE [0]. Button})) is subtracted to obtain the maximum data size.

The maximum data size calculated in the above manner is divided by thetransfer rate of the playback apparatus (128 Mbps), and furthermultiplied by 90 KHz for expressing the result in the time accuracy ofPTS (90 KHz). Then, the transfer duration taken for presentation of thefirst page of the Multi-Page menu is calculated. FIG. 35C shows theequation for calculating the PAGENODEFAULTTD (DS_(n)) in the case whereno in-effect is provided and the default selected button is dynamicallydetermined.

FIG. 36 is a view showing an algorithm for selectively determining oneof the EFFECTTD, PAGEDEFAULTTD, and PAGENODEFAULTTD as theTRANSFERDURATION. As shown in the figure, if at least one in-effect isprovided (if (DS_(n)[ICS].PAGE[0].IN EFFECTS.number_of effects!=0)), theEFFECTTD is determined as TRANSFERDURATION (return EFFECTTD(DS_(n))). Ifno in-effect is provided for introducing Page[0] and no default selectedbutton is statically specified (DS_(n)[ICS.PAGE[0].default_selected_button_id_ref==0xFFFF]) the PAGEDEFAULTTD isdetermined as the TRANSFERDURATION (return PAGENODEFAULTTD(DS_(n))). Ifno in-effect is provided for introducing Page[0] and the defaultselected button is statically determined, the PAGEDEFAULTTD isdetermined as the TRANSFERDURATION (return PAGEDEFAULTTD (DS_(n))).

As described above, according to the fifth embodiment of the presentinvention, the DTS and PTS values of an ICS are optimally determined inaccordance with the data size of ODSs. Thus, the interactive control issmoothly performed without delay.

Sixth Embodiment

A sixth embodiment of the present invention relates to the playbackapparatus used for playback of the BD-ROMs described in the first tofifth embodiments. FIG. 37 is a view showing the internal structure ofthe playback apparatus. The playback apparatus according to the sixthembodiment is industrially manufactured based on the internal structureshown in FIG. 37. The playback apparatus is mainly composed up of twoparts that are a system LSI and a drive device. The playback apparatuscan be industrially manufactured by mounting these parts on a cabinetand substrate of the apparatus. The system LSI is an integrated circuitincluding various processing units for achieving the functions of theplayback apparatus.

The playback apparatus manufactured in the above manner is composed of aBD drive 1, a read buffer 2, a demultiplexer (De-MUX) 3, a video decoder4, a video plane 5, P-Graphics decoder 6, a presentation plane 7, anadder 8, an I-Graphics decoder 9, an interactive graphics plane 10, anadder 11, an audio decoder 13, CLUT (Color Lookup Table) units 14 and15, a PRS set 16, a system clock 17, a uses timeout timer 18, aselection_timeout timer 19, a compostion_timeout timer 20, aneffect_duration timer 21, an operation receiving unit 22, and a statecontrol unit 23.

The BD drive 1 performs loading, reading, and ejecting of the BD-ROM.The BD drive 1 makes access to the BB-ROM 100.

The read buffer is a FIFO (first-in first-out) memory. Accordingly, TSpackets read from the BD-ROM are removed from the read buffer 2 in thesame order as they arrive.

The De-MUX 3 reads TS packets from the read buffer 2 and converts themto PES packets. Among the resulting PES-packets, PES packets having apredetermined stream PID are outputted to one of the video decoder 4,the audio decoder 13, the P-Graphics decoder 6, and the I-Graphicsdecoder 9.

The video decoder 4 decrypts PES packets received from the De-MUX 3, andrendered the resulting uncompressed picture data to the video plane 5.

The video plane 5 is a plane memory for storing uncompressed picturedata. The term “plane” refers to a memory area of a playback apparatusfor storing one screen of pixel data. With provision of multiple planes,the playback apparatus can add the stored data on each plane pixel bypixel to produce one video output, whereby a plurality of graphicsimages are overlaid as one composite image. The resolution of the videoplane 5 is 1920×1080. Picture data is stored on the video plane 5 as16-bit pixel data expressed in YUV values.

The P-Graphics decoder 6 decodes a graphics stream read from the BD-ROMor HD, and renders the resulting uncompressed graphics to thepresentation graphics plane 7. As the graphics stream is decoded andrendered, subtitles appear on screen.

The presentation graphics plane 7 is a memory having an area for storingone screen of uncompressed graphics. The resolution of this plane is1920×1080, and uncompressed graphics are stored on the presentationgraphics plane 7 as 8-bit pixel data expressed in index color. Byconverting the index color using the CLUT (Color Lookup Table), theuncompressed graphics stored on the presentation graphics plane 7 arepresented.

The adder 8 overlays the uncompressed picture data (i) with the contentsstored on the presentation graphics plane 7.

The I-Graphics decoder 9 decodes an IG stream read from the BD-ROM orHD, and renders the resulting uncompressed graphics to the interactivegraphics plane 10.

To the interactive graphics plane 10, uncompressed graphics data decodedby the I-Graphics decoder 9 is rendered.

The adder 11 overlays the data on the interactive graphics plane 10 withthe output of the adder 8 (i.e. with the interim video data maid byoverlaying uncompressed picture data and the contents stored on thepresentation graphics plane 7) to make the final video output.

The audio decoder 13 decodes PES packets received from the De-MUX 3 tooutput uncompressed audio data.

The CLUT unit 14 converts index colors of the uncompressed graphicsstored on the presentation graphics plane 7 to Y, Cr, and Cb values.

The CLUT unit 15 converts index colors of the uncompressed graphicsstored on the interactive graphics plane 10 to Y, Cr, and Cb values.

The PSR set 16 is a set of registers provided within the playbackapparatus. The set of registers include 64 player status registers(PSRs) and 4,096 general purpose registers (GPRs). The 64 player statusregisters represent the current status of the playback apparatus.Specifically, the PSRs 5-8 represent the current playback point. The PRS8 takes on a value ranging from 0 to 0xFFFFFFFF representing the currentplayback point (current PTM: Presentation Time) in 45 KHz. The PSR 11represents the page_id of the currently active page (current page). ThePSR 10 represents the button in the selected state (current button) onthe current page.

The system clock 17 generates clock signals. The De-MUX 3 performs theconversion of PES packet in synchronism with the clock signals.

The user_timeout timer 18 is set to the value of user_time_out_durationfield at the time specified by the PTS (DS_(n)[ICS]). When the user_timeout_duration field value is set, the timer 18 starts to count downtoward zero in system clock ticks. The timer 18 is reset back to theuser_time_out_duration field value each time a user operation isreceived. If no user operation is received before the timer 18 reacheszero, the timer 18 times out.

The selection_timeout timer 19 is set to the value ofselection_time_out_pts field at the time specified by the PTS(DS_(n)[ICS]). When the selection_time_out_pts field value is set, thetimer 19 starts to count down toward zero in system clock ticks. Thetimer 19 is reset back to the selection_time_out_pts field value eachtime a user operation is received. If no user operation is receivedbefore the timer 19 reaches zero, the timer 19 times out. When the timer19 times out, it is known that the valid interaction period shown inFIG. 9 has ended.

The composition_timeout timer 20 is set to the value ofcompostion_time_out_pts field at the time specified by theDTS(DS_(n)[ICS]). When the compostion_time_out_pts field value is set,the timer 20 starts to count down toward zero in system clock ticks. Thetimer 20 is not reset even if a user operation is received. When thetimer 20 reaches to zero, i.e. the timer 20 times out, it is known thatthe Epoch END has been reached.

The effect_duration timer 21 is set to the value of effect_durationfield at the time specified by the DTS (DS_(n)[ICS]) if the DS_(n) is anEpoch Start DS. When the effect_duration field value is set, the timer21 starts to count down toward zero in system clock ticks. The timer 21is not reset even if a user operation is received. When the timer 21reaches zero, i.e. the timer 21 times out, it is known a next displaycomposition of the animation effect needs to be displayed.

The operation receiving unit 22 receives a user operation made on theremote controller or the front panel of the playback apparatus, andoutputs information representing the received user operation (theinformation is hereinafter referred to as “UO”) to the I-Graphicsdecoder 9.

The state control unit 23 sets desired values to the PSRs according tothe instructions from the I-Graphics decoder 9. Such instructions may begiven by way of (i) direct addressing or (ii) indirect addressing. Inthe case of direct addressing, the I-Graphics decoder 9 outputs animmediate value to be set to an appropriate register of the PSR set 16.In the case of indirect addressing, the I-Graphics decoder 9 outputs anotification of a change in the playback apparatus status or in the userpreferences. On receiving such a notification, the status controller 23determines the value to be set to reflect the change, and sets the thusdetermined value to an appropriate register of the PSR set 16. Thestatus controller 23 determines the value by performing a “procedurewhen playback condition is changed” or “procedure when change isrequested”. Hereinafter, a description is given to the proceduresperformed for setting the PSR 11 (current page) and the PSR 10 (currentbutton).

FIG. 38A is a flowchart of the “procedure when playback condition ischanged”, preformed with respect to the PSR 11. This procedure isperformed to set the first page_info structure within an ICS to the PSR11 (step S99).

FIG. 38B is a flowchart of the “procedure when change is requested”,preformed with respect to the PSR 11. Suppose that a user operation isreceived and a page having the page number X is requested to bepresented. In response to the user operation, the status controller 23judges whether X is a valid value (step S100). If X is a valid value,the PSR 11 is set to the value of X (step S101). If X is invalid, thePSR 11 remains unchanged (step S102).

The value of PSR 11 changes as described above. Turning now to the PSR10, a description is given to the “procedure when playback condition ischanged” and to the “procedure when change is requested”.

FIG. 39A is a flowchart of the “procedure when playback condition ischanged”, performed with respect to the PSR 10.

First, the status controller 23 judges whether thedefault_selected_button_id_ref field value associated with the currentpage is valid (step S111). If the judgment in the step S111 results inYES, the default_selected_button_id_ref field value is set to the PSR 10(step S112)

If the default selected button_id_ref field value is invalid, it is thenjudged whether the PSR 10 holds a valid value (step S113). If the PSR 10is valid, the PSR 10 remains unchanged so that the currently held valueis maintained (step S114). On the other hand, if the PSR 10 is invalid,the PSR 10 is set to the button_id_ref field value identifying the firstbutton_info structure of the current page (step S115).

FIG. 39B is a flowchart of the “procedure when change is requested”,performed with respect to the PSR 10. Suppose that a user operation ismade and a button having the button number X is requested to beselected. In response to the user operation, the status controller 23judges Whether X is a valid button_id field value (step S116). If X is avalid value, the PSR 10 is set to the value of X (step S118). If X is aninvalid value, the PSR 10 remains unchanged and thus the currently heldvalue is maintained (step S117). With the above-described procedures,the PSRs 10 and 11 are ensured to hold current values at all times. Thisconcludes the description about the internal structure of playbackapparatus.

<Internal Structure of I-Graphics Decoder 9>

Next, a description is given to the internal structure of I-Graphicsdecoder 9, with reference to FIG. 40. As shown in the figure, theI-Graphics decoder is composed of a coded data buffer 33, a streamgraphics processor 34, an object buffer 35, a composition buffer 36, anda graphics controller 37.

The coded data buffer 33 is a buffer for temporally storing an ICS, aPDS, and an ODS along with their DTS and PTS.

The stream graphics processor 34 decodes an ODS and outputs theresulting uncompressed graphics objects to the object buffer 35.

The object buffer 35 is a buffer for storing a plurality of uncompressedgraphics objects (represented by the boxes in the figure), which aredecoded by the stream graphics processor 34.

The composition buffer 36 is a memory for storing an ICS. Thecomposition buffer 36 supplies to the graphics controller 37, thepage_info structures provided within the ICS and the button_infostructures provided within each page_info structure.

The graphics controller 37 decodes an ICS stored in the compositionbuffer 37 and assembles a display composition based on the decoded ICS.That is, the graphics controller 37 reads graphics objects from theobject buffer 15 with reference to the page_info structure identified bythe value of PSR 11 (current page_info structure), and renders the readgraphics objects to the interactive graphics plane 10. The graphicsobjects read herein are a sequence of graphics objects specified by thenormal_start_object_id_ref field value through thenormal_end_object_id_ref field value of each button_info structureassociated with the current page. With respect to the button_infostructure specified by the PSR 10, a sequence of graphics objectsspecified by the selected_start_object_id_ref field value through theselected_end_object_id_ref field value is read from the object buffer15. In FIG. 40, arrows bg1, bg2, bg3, and bg4 schematically representthe rendering of graphics objects performed by the graphics controller37. As a result of the rendering, a page including the buttons O-A, O-B,O-C, and O-D is stored on the interactive graphics plane 10 and overlaidwith the video.

This concludes the rough description of procedures to be performed bythe graphics controller 37. Furthermore, the graphics controller 37updates the display composition in response to the changes in the valueof PSR 10 or PSR 11, the timeout of the timers 18-21, or a useroperation received by the composition timeout timer 20. The proceduresperformed by the graphics controller 37 are shown in FIGS. 41-47 indetail.

FIG. 41 is a flowchart of the main routine of the procedure performed bythe graphics controller 37. In the main routine, steps S1-S6 form aloop. In each iteration of the loop, judgments are made as to whetherspecific events have occurred (steps S1-S4), a display composition ofthe Multi-Page menu is updated to a subsequent display composition foranimated presentation (step S5), and processing is performed accordingto a user operation (UO processing) (step S6). If any of the judgmentsin the steps S1-S4 results in YES, corresponding steps are performed andthe processing returns to the main routine.

In the step S1, the judgment is made as to whether the current playbackpoint has reached the point specified by the PTS (DS_(n)[ICS]). If thejudgment in the step S1 results in YES, the above described “procedurewhen playback condition is changed” is performed. As a result, Page[0],which is the 1^(st) page of the Multi-Page menu, is set as the currentpage (step S7). Then, the user_timeout timer 18, the selection_timeouttimer 19, and the composition_timeout timer 20 are all reset (step S8),and the current page is presented on screen (step S9). The processingthen goes back to the step S1 for the next loop iteration.

In the step S2, the judgment is made as to whether the user_timeouttimer 18 has timed out. If the judgment in the step S2 results in YES, aloop of steps S11-S14 is performed. In each iteration of the loop, anarbitrary Page[j] of the Multi-Page menu is removed from the screen(step S13), followed by playback of an out-effect animation provided forPage[j] (step S14). The loop is repeated for each page from Page[0]onward, defined within the ICS (steps S11 and S12).

In the step S3, the judgment is made as to whether the selection_timeouttimer 19 has timed out. If the judgment in the step S3 results in YES, abutton on the current page is automatically activated (auto-activate:steps S1). Then, the processing returns the step S1 for the next loopiteration.

In the step S4, the judgment is made as to whether the compositiontimeout timer 20 has timed out. If the judgment in the step S4 resultsin YES, Page[0], which is the first page of the Multi-Page menu, isremoved from the screen (step S15), followed by playback of theout-effect animation provided for Page[0] (step S16). Then, theprocessing returns to the step S1 for the next loop iteration.

Now, a description is given to the first presentation of the multi-pagemenu. The first presentation is carried out through steps S15-S23 shownin FIG. 42, after the state control information 23 performs theabove-described “procedure when playback condition is changed” to setPage[0] as the current page. In the step S24, the in-effect animationfor the current page is executed. In the step S25 that follows, acurrent button setting procedure is performed. The current button isspecified by the PSR 10. Thus, in the step S25, the “procedure whenplayback condition is changed” is performed with respect to the PSR 10,so that the current button is determined. Then, the processing movesonto the steps S17-S22.

The steps S17 and S22 form a loop which is repeated for each button_infostructure provided in the current page (steps S17 and S18). Thebutton_info structure to be processed in the current iteration of theloop is designated as button_info (p).

In the step S19, a judgment is made as to whether the button_info (p) isassociated with the current button. If the judgment in the step S19results in YES, the step S20 is performed next. If not, the step S21 isperformed next.

In the step S21, among the graphics objects on the object buffer 35, onespecified by the normal_start_object_id_ref field value is designated asthe graphics object (p).

In the step S20, among the graphics objects on the object buffer 35, onespecified by the selected_start_object_id_ref field value is designatedas the graphics object (p).

Once the graphics object (p) is designated in the step S20 or S21, thegraphics object (p) is rendered to the interactive graphics plane 10 atthe position specified by the button_horizontal_position andbutton_vertical_position field values (step S22). By repeating the abovesteps for each button_info structures provided in the current page,among a plurality of graphics objects associated with each button state,the first graphics object is rendered for each button to the interactivegraphics plane 10. After repeating the above steps, the CLUT unit 15 isset so as to present the current page using the pallet data specified bythe pallet_id_ref field value of the current page (step S23). Thisconcludes the description of the flowchart shown in FIG. 42.

FIG. 43 is a flowchart of the procedure for playback of an in-effectanimation. In the flowchart, a variable t denotes an arbitrary one ofdisplay compositions within the effect_sequence defined for thein-effect. In addition, a variable u denotes an arbitrary one of acomposition_object out of composition_objects used for the effect(t). Asshown in the flowchart, first of all, the variables t and u areinitialized to “0” (step S30). Then, the effect_duration_timer 21 is setto the value of the effect (t).effect_duration field (step S31), and theCLUT units 14 and 15 are set so as to use the pallet data identified bythe effect (t).pallet_id_ref field value for presentation (step S32).Then, a loop of steps S33-S40 is performed. The loop is a two-levelloop. The first-level loop (steps S33-S37) uses the variable u as acontrol variable, whereas the second-level loop (step S33-S40) uses thevariable t as a control variable.

In the first-level loop, a judgment is made as to whether theobject_cropped_flag field provided within the composition_object (u)structure associated with the effect (t) is set to “0” (step S33). Ifthe field is set to “0”, no graphics object is presented on screen forcomposition_object (u) (step S35). If the field is set to “1”, on theother hand, the object is cropped according to theobject_cropping_horizontal_position, object_cropping_vertical_position,cropping width, and cropping_height fields of the composition_object(u). The cropped part of the graphics object is then rendered within thewindow identified by the window_id_ref field of thecomposition_object(u), at the position specified by thecomposition_object_horizontal_position and thecomposition_object_vertical_position fields within thecomposition_object(u) (step S34). Then, the variable u is incremented by“1”. The first-level loop is repeated until the variable u is equal tothe number_of_composition_object field value.

In the second-level loop, for each iteration of the first-level loop(steps S33-S37), the following steps are performed. After theeffect_duration times out (step S38), the variable t is incremented by“1”, and the variable u is initialized to “0” (steps S39). Thesecond-level loop is repeated until the variable t is equal to thenumber_of_effect (t) field value (step S40). This concludes thedescription of the procedure for playback of In-effect. Note that nodescription is given to the procedure for playback of an out-effect,because it is basically the same to the above-described procedure.

Immediately after playback of the in-effect, a current page is presentedand the first display is updated for animated presentation. The firstdisplay composition of the current page is presented by rendering to theinteractive graphics plane 10, the graphics objects associated with thenormal_start_object_id_ref and the selected_start_object_id_ref fieldvalues provided in each button_info structure. Each button is presentedin animation by updating the button image on the interactive graphicsplane 10 for each iteration of the loop in the main routine. Each buttonimage is updated by overwriting the interactive graphics plane 10 withan arbitrary one of a sequence of graphics objects (qth graphics object)associated with the button. That is to say, each button is presented inanimation by rendering a sequence of graphics objects associated withthe normal_state_info and selected_state_info fields provided withineach button_info structure, one by one to the interactive graphics plane10. Note that the variable q is used to specify individual graphicsobjects associated with the normal_state_info and selected-state_infofields provided within each button_info structure.

Now, with reference to FIG. 44, a description is given to the procedurefor display update for animated presentation.

In a step S41, a judgment is made as to whether the first displaycomposition has been presented. If the first display composition is notyet presented, the processing returns to the main routine withoutperforming any operation. If the first display composition has beenpresented, on the other hand, steps S42-S53 are performed. The stepsS41-S55 form a loop in which the steps S44-S55 are repeated for eachbutton_info structure in the ICS (steps S42 and S43).

In the step S44, the variable q is set to the value of variableanimation (p) corresponding to the button_info (p) structure. As aresult, the variable q represents the current frame number of thebutton_info(p).

In the step S45, a judgment is made as to whether the button_info(p)corresponds to the button currently in the selected state (i.e. thecurrent button).

If the button_info(p) does not correspond to the current button, thevariable q is added to the normal_start_object_id_ref in thebutton_info(p) to obtain an ID(q) (step S46).

If the button_info (p) corresponds to the current button, the step S47is performed.

In the step S47, a judgment is made as to whether the current button isin the activated state. If the judgment in the step S47 results in YES,the variable q is added to the activated_start_object_id_ref field valuein the button_info(p) to obtain an ID(q) (step S54). Then, one of thenavigation commands associated with the button_info (p) is executed(step S55).

On the other hand, if the current button is not in the activated state,the variable q is added to the selected_start_object_id_ref field valuein the button_info(p) to obtain an ID(q) (step S48).

Once the ID(q) is obtained, among the graphics objects stored on theobject buffer 35, the one identified by the ID (q) is rendered to theinteractive graphics plane 10. The rendering position on the interactivegraphics plane 10 is specified by the button_horizontal_position and thebutton_vertical_position fields provided in the button info (p).

Through the loop iteration, the q^(th) one of the graphics objectsassociated with the selected (or activated) state of the current buttonis rendered to the interactive graphics plane 10.

In the step S50, a judgment is made as to whether the sum of thenormal_start_object_id_ref field value and the variable q has reachedthe normal_end_object_id_ref field value. If the sum has not reached thenormal_end_object_id_ref field value, the variable q is incremented by“1” and the incremented variable q is set as the variable animation (p)(step S51). On the other hand, if the sum has reached thenormal_end_object_id_ref field value, a judgment is made as to whetherthe repeat_flag field is set to “1” (step S52). If the repeat_flag fieldis set to “1”, the variable animation (p) is initialized to “0” (stepS53). The above steps are repeated for each button_info structure withinthe ICS (steps S42 and S43), and then processing returns to the mainroutine.

Through the above-described steps of S41-S53, each button imagepresented on screen is updated with a new graphics object each time themain routine is performed. Thus, by repeating the main routine, thebuttons images are presented in animation. When presenting the buttonsin animation, the graphics controller 37 adjusts the duration forpresenting each graphics object so as to maintain theanimation_frame_rate_code. This concludes the description of theprocedure for animated presentation.

In the step S47 described above, if it is judged that the current buttonis in the activated state, a navigations command associated with thecurrent button needs to be executed in the step S55. FIG. 45 is aflowchart of the procedure for executing a navigation command. First, ajudgment is made as to whether the navigation command is aSetButtonPageCommand (step S56). If the navigation command is not aSetButtonPageCommand, the navigation command is simply executed (stepS57). On the other hand, if the navigation command is aSetButtonPageCommand, instructions are given to the state control unit23 so as to designate the page number specified by the operand of thenavigation command as the page number X as well as to set the buttonnumber specified by the operand of the navigation command to the PSR 10(step S58). Then, the state control unit 23 executes the “procedure whenchange is requested”, with respect to the PSR 11 (step S59). Asmentioned above, the PSR 11 holds a value representing the currentlypresented page (i.e. the current page). Thus, by performing the“procedure when change is requested” with respect to the PSR 11, thecurrent page is determined. Then, the state control unit 23 performs the“procedure when playback condition is changed”, with respect to the PSR10 (step S60). This concludes the description of the flowchart shown inFIG. 45.

This concludes the description of the procedure for executing anavigation command. As stated above, a navigation command is executedwhen an associated button is activated. The button state changes arepresented through the following procedure in response to a useroperation (UO). FIG. 46 shows the procedure for processing a UO. Asshown in the flowchart, judgments are made in steps S61-S64 as towhether specific events have occurred. An occurrence of each eventrequires specific steps to be performed. The processing then returns tothe main routine. Specifically, in the step S61, a judgment is made asto whether the UO_mask_table field is set to “1”. If the field is set to“1”, the processing returns to the main routine without performing anystep.

In the step S62, the judgment is made as to whether any of the Move Up,Down, Left, and Right keys on the remote controller is pushed. At a pushof any of the keys, the user_timeout timer 18 and the selection_timeouttimer 19 are reset (step S65). Then, another button is designated as thenew current button (step S66), followed by the judgment as to whetherthe auto_action_flag of the newly designated current button is set to“01” (step S67). If the auto_action_flag is not set to “01”, theprocessing returns to the main routine. On the other hand, if theauto_action_flag is set to “01”, the current button(i) is activated(step S69). Then, the variable animation(i) is set to “0” (step S70).

In the step S63, the judgment is made as to whether a numeric input ismade. If a numeric input is made, the user_timeout timer 18 and theselection timeout timer 19 are reset (step S71), and another button isdesignated as the new current button (step S72). The processing thenreturns to the main routine.

In the step S64, a judgment is made as to whether an activate key on theremote controller is pushed. At a push of the activate key, theuser_timeout timer 18 and the selection_timeout timer 19 are reset (stepS68), and then the current button (i) is activated (step S69). After thebutton state transition, the variable animation(i) is set to “0” (stepS70). Note that the step S66 of designating a new current buttondescribed above is performed by calling a subroutine shown in FIG. 47.Now, a description is given to the subroutine, with reference to thefigure.

FIG. 47 is a flowchart of the current button change procedure. First, itis determined which of upper_button_id_ref, lower_button_id_ref,left_button_id_ref, and right_button_id_ref fields within theneighbor_info associated with the current button corresponds to thepushed key (step S75).

Here, let a button(Y) denote the current button, and let a button Xdenote the button specified by the one of upper-button_id_ref,lower_button_id_ref, left_button_id_ref, and right_button_id_ref fields(step S76). Then, the state control unit 23 performs the “procedure whenchange is requested” with respect to the PSR 10 (step S77). As a result,the PSR 10 is set to the value X.

Thereafter, the variables animation (X) and animation(Y) are set to “0”(step S78), and returns to the main routine.

As described above, in the UO processing, a button is activated inresponse to a UO. Yet, the button state transition to the activatedstate is made also when the selection_time_out_pts times out. Now, withreference to FIG. 48, a description is given to the procedure forauto-activation of a button upon timeout of the selection_time_out_pts.

FIG. 48 is a flowchart of the procedure for auto-activation 20 of abutton. First, a judgment is made as to whether thedefault_activated_button_id_ref field is set to “00” or “FF” (step S91).If the field is set to “00”, the processing returns to the main routinewithout performing any step. On the other hand, if thedefault_activated_button_id_ref is set to “FF”, the current button(i) isactivated (step S93). Thereafter, the variable animation(i)corresponding to the current button(i) is set to “0” (step S94), andreturns to the main routine.

If the default_activated_button_id_ref field value is neither “00” nor“FF”, the button specified by the default_activated_button_id_ref fieldvalue is determined to be the new current button (i) (step S92). Thus,the button specified by the default_activated_button_id_ref field valuereceives the activated state (step S93). Thereafter, the variableanimation (i) corresponding to the current button(i) is set to “0” (stepS94), and returns to the main routine.

Through the above steps, the button in the selected state isautomatically put into the activated state at an end of a predeterminedduration. This concludes the description of the flowchart shown in FIG.47.

<Turn On/Off of Graphics Display in Pop-Up U/I>

The description given above relates to the procedure performed by theI-Graphics decoder 9 in the case where the user_inderface_model showsthe Always-On U/I. In the case where the Pop-Up U/I is used, theI-Graphics decoder 9 operates as follows. First, similarly to theAlways-On U/I case, the I-Graphics decoder 9 performs the decoding toobtain decoded graphics objects on the object buffer 35. The I-Graphicsdecoder 9 then renders the graphics objects to the interactive graphicsplane 10 by the time shown by the PTS(DS_(n)[ICS]). Up to this point,the I-Graphics decoder 9 operates in a similar manner to the case whereAlways-On U/I is used. Yet, the processing performed thereafter differsin that the graphics controller 37 does not overlay the page stored onthe interactive graphics plane 10, and thus the page is not presented.That is to say, graphics controller 37 “turns off” the page by notoutputting the data stored on the interactive graphics plane 10 to theCLUT unit 15. With the page not presented on screen (i.e. turned off),the graphics controller 37 waits for a user operation for Pop-Up_on. Inresponse to a user operation for Pop-Up_on, the graphics controller 37outputs the graphics data stored on the interactive graphics plane 10 tothe CLUT unit 15 where the graphics data is overlaid with the videodata. As a result, the page is “turned on” or presented on screen. Asabove, the graphics controller 37 turns on and off the graphics displayto realize Pop-Up display.

In the operations performed thereafter, there is no difference betweenthe Always On U/I and the Pop-Up U/I. When any of the buttons on themain page is activated, a navigation command associated with theactivated button is executed. When the selection_timeout timer 19 timesout, the current button on the main page is automatically activated anda sub page is presented on screen.

When a user operation is made for Pop-Up_off, the graphics controller 37tunes off all on-screen pages, thereby leaving no page presented onscreen.

If no user operation is made for a certain period of time, theuser_timeout timer 18 counts down toward zero. When the user_timeouttimer 18 times out, the graphics controller 37 also turns off allon-screen pages, thereby leaving no page presented on screen. The turnon and turn off procedures described above are performed to realize thestate transition shown in FIG. 12A.

As above, through the turn on and the turn off operations, Pop-Up U/I isimplemented.

As described above, the sixth embodiment of the present inventionrealizes the industrial production of playback apparatuses supportingthe BD-ROM having the data structures as described in the first to fifthembodiments. This helps for widespread use of the BD-ROMs according tothe first to fifth embodiments.

Seventh Embodiment

A seventh embodiment of the present invention relates to a manufacturingprocess of the BD-ROM. FIG. 49 is a flowchart of the manufacturingprocess of the BD-ROMs according to the first to fifth embodiments.

The manufacturing process includes a material production step ofrecording video, audio, and the like (step S201), an authoring step ofcreating an application format using an authoring device (step S202),and a pressing step of creating an original master of the BD-ROM andperforming stamping and bonding to complete the BD-ROM (step S203).

The BD-ROM authoring step includes the following steps S204-S213.

Now, a description is given to the steps S204-S213. In the step S240,control information, palette definition information, and graphics aregenerated. In the step S205, the control information, the palettedefinition information, and the graphics are converted to functionalsegments. In the step S206, a PTS of each ICS is set based on thedisplay timing of a picture to be synchronized with. In the step S207, aDTS [ODS] and a PTS[ODS] are set based on the PTS[ICS]. In the stepS208, a DTS[ICS] and a PTS[PDS] are set based on the DTS[ODS].

In the step S209, changes in occupancy of each buffer in the playermodel are graphed. In the step S210, a judgment is made as to whetherthe graphed changes satisfy the constraints of the player model. If thejudgment results in the negative, the DTS and PTS of each functionalsegment are rewritten in the step S211. If the judgment results in theaffirmative, a graphics stream is generated in the step S212, and thegraphics stream is multiplexed with a video stream and an audio stream,which are separately generated, to form an AV Clip in the step S213.After this, the AV Clip is adapted to the Blue-ray Disc Read-OnlyFormat, to complete the application format.

Modifications

Though the Present Invention has been Described by Way of the aboveembodiments, the present invention is not limited to those specificembodiments. The present invention may be embodied with any of themodifications (A) to (L) below. The invention recited in each of theclaims of this application includes extension and generalization of theabove embodiments and their modifications below. The degree of extensionand generalization depends upon the state of the art in the technicalfield of the present invention at the time when the present inventionwas made.

(A) The above embodiments describe the case where the BD-ROM is used asthe recording medium. Main features of the present invention, however,lie in a graphics stream recorded on the recording medium, which doesnot rely on physical characteristics of BD-ROMS. Therefore, the presentinvention is applicable to any recording medium that is capable ofrecording a graphics stream. Examples of such a recording mediuminclude: an optical disc such as a DVD-ROM, a DVD-RAM, a DVD-RW, aDVD-R, a DVD+RW, a DVD+R, a CD-R, or a CD-RW; a magneto-optical disksuch as a PD or an MO; a semiconductor memory card such as aCompactFlash card, a SmartMedia card, a Memory Stick card, aMultimediacard, or a PCMCIA card; a magnetic disk such as a flexibledisk, SuperDisk, Zip, or Clik!; a removable hard disk drive such as ORB,Jaz, SparQ, SyJet, EZFley, or Microdrive, and a nonremovable hard diskdrive.

(B) The above embodiments describe the case where the playback apparatusdecodes an AV Clip on the BD-ROM and outputs the decoded AV Clip to thetelevision. As an alternative, the playback apparatus may be equippedwith only a BD drive, with the remaining construction elements beingprovided in the television. In this case, the playback apparatus and thetelevision can be incorporated in a home network connected with an IEEE1394 connector. The above embodiments describe the case where theplayback apparatus is connected to the television, but the playbackapparatus may instead be integrated with a display device. Also, theplayback apparatus may include only the system LSI (integrated circuit)which carries out an essential part of the processing. The playbackapparatus and the integrated circuit are both an invention described inthis specification. Accordingly, regardless of whether the playbackapparatus or the integrated circuit is concerned, an act ofmanufacturing a playback apparatus based on the internal structure ofthe playback apparatus described in the sixth embodiment is an act ofpracticing the present invention. Also, any act of assigning with charge(i.e. for sale) or without charge (i.e. as a gift), leasing, andimporting the playback apparatus is an act of practicing the presentinvention. Likewise, an act of offering for assignment or lease of thereproduction apparatus using storefront displays, catalogs, or brochuresis an act of practicing the present invention.

(C) Information processing using the programs shown in the flowcharts isactually realized using hardware resources. Accordingly, the programswhich describe the procedure steps shown in the flowcharts arethemselves an invention. The above embodiments all relate to the casewhere the programs are incorporated in the playback apparatus, but theprograms can be used independently of the playback apparatus. Acts ofpracticing the programs include (1) an act of manufacturing, (2) an actof assigning with or without charge, (3) an act of leasing, (4) an actof importing, (5) an act of providing to the public via a bi-directionalelectronic communications network, and (6) an act of offering forassignment or lease using storefront displays, catalogs, or brochures.

(D) The time elements of the steps which are executed in a time seriesin each of the flowcharts can be regarded as the necessary elements ofthe present invention. This being so, a playback method shown by theseflowcharts is an invention. If the processing shown in each flowchart iscarried out by performing the steps in a time series so as to achievethe intended aim and the intended effect, it is considered as an act ofpracticing the playback method of the present invention.

(E) When recording an AV Clip on the BD-ROM, an extension header may beadded to each TS packet in the AV Clip. The extension header is called aTP_extra_header, includes an arrival_time_stamp and acopy_permission_indicator, and has a data length of 4 bytes. TS packetswith TP_extra_headers (hereafter “EX TS packets”) are grouped in unitsof 32 packets, and each group is written to three sectors. One groupmade up of 32EXTS packets has 6,144 bytes (=32×192), which is equivalentto a size of three sectors that is 6144 bytes (=2048×3). The 32 EX TSpackets contained in the three sectors are called an “Aligned Unit”.

In a home network connected with an IEEE 1394 connector, the playbackapparatus transmits an Aligned Unit in the following manner. Thereproduction apparatus removes a TP_extra_header from each of the 32 EXTS packets in the Aligned Unit, encrypts the body of each TS packetaccording to the DTCP Specification, and outputs the encrypted TSpackets. When outputting the TS packets, the playback apparatus insertsan isochronous packet between adjacent TS packets. A position where theisochronous packet is inserted is based on a time shown by anarrival_time_stamp of the TP_extra_header. The playback apparatusoutputs a DTCP_descriptor, as well as the TS packets. TheDTCP_descriptor corresponds to a copy_permission_indicator in theTP_extra_header. With the provision of the DTCP_descriptor indicating“copy prohibited”, it is possible to prevent, when using the TS packetsin the home network connected via the IEEE 1394 connector, the TSpackets from being recorded to other devices.

(F) The above embodiments relate to the case where an AV Clip of theBlu-ray. Disc Read-Only Format is used as a digital stream, but thepresent invention may also be embodied with a VOB (Video Object) of theDVD-Video Format or the DVD-Video Recording Format. The VOB is a programstream that complies with the ISO/IEC 13818-1 Standard and is obtainedby multiplexing a video stream and an audio stream. Also, the videostream in the AV Clip may be an MPEG4 video stream or a WMV videostream. Further, the audio stream in the AV Clip may be a Linear PCMaudio stream, a Dolby AC-3 audio stream, an MP3 audio stream, anMPEG-AAC audio stream, or a dts audio stream.

(G) The movie described in the above embodiments may be one obtained byencoding an analog image signal broadcast by analog broadcasting. Also,the movie may be stream data made up of a transport stream broadcast bydigital broadcasting.

Alternatively, an analog/digital image signal recorded on a videotapemay be encoded to obtain content. Also, an analog/digital image signaldirectly captured by a video camera may be encoded to obtain a content.A digital work distributed by a distribution server is applicable too.

(H) Graphics objects described in the above embodiments is run-lengthencoded raster data. Run-length encoding is used forcompression/encoding of graphics objects, because the run-lengthencoding is suitable for compression and decompression of subtitles.Subtitles have the property that a continuous length of the same pixelvalue in a horizontal direction is relatively long. Therefore, byperforming compression using run-length encoding, a high compressionrate can be attained. In addition, run-length encoding reduces a loadfor decompression, and is therefore suitable for realizing decoding bysoftware. Nevertheless, the use of run-length encoding for graphicsobjects is not essential to the present invention. For example, graphicsobjects may be PNG data. Also, graphics objects may be vector datainstead of raster data. Further, graphics objects may be transparentpatterns.

(I) The transfer rate Rc may be determined so that clearing andrendering of the graphics plane complete within a vertical blankingtime. Suppose the vertical blanking time is 25% of 1/29.93 seconds. ThenRc is determined to be 1 Gbps. By determining Rc in this way, graphicscan be displayed smoothly, which achieves a significant effect on thepractical use.

Also, writing in sync with line scan can be used together with writingwithin a vertical blanking time. This ensures smooth presentation if thetransfer rate Rc is 256 Mbps.

(J) The above embodiments relate to the case where the playbackapparatus is provided with the graphics plane. Alternatively, theplayback apparatus may include a line buffer for storing uncompressedpixels of one line. Since conversion to an image signal is performed foreach horizontal row (line), conversion to an image signal can equally beperformed with the line buffer.

(K) In the case of a merge point of multiple playback paths, a differentbutton needs to be selected as default depending on a playback pathhaving been taken. Thus, it is desirable to define such playback controlin the static scenario that a value unique to each playback path isstored to a register of the playback apparatus at the time when theplayback path is actually taken. The playback procedure may be set toput the button specified by the register value to the selected state. Inthis way, a different button is put to the selected state depending on aplayback path having been taken.

(L) The graphics plane described in the sixth embodiment preferably hasa double buffer structure. If the graphics plane has a double bufferstructure, even when large-sized graphics data needs to be rendered,which requires a time corresponding to several frames, each displaycomposition may be instantaneously presented by switching between twobuffers. Thus, a double buffer structure is effective when the menu of afull-screen size needs to be presented.

INDUSTRIAL APPLICABILITY

The playback apparatus according to the present invention is suitablefor a personal use, for example in a home theater system. Since theabove embodiments of the present invention disclose the internalstructure, playback apparatuses having the internal structure can beproduced on a large scale. Thus, the playback apparatus according to thepresent invention is industrially usable in its quality and hasindustrial applicability.

1. A recording medium having recorded thereon a graphics stream,wherein: the graphic stream includes one or more interactive compositionsegments and object definition segments, each object definition segmentdefining a graphics object; each interactive composition segmentincludes time information, stream model information, and a plurality ofpieces of page information; the plurality of pieces of page informationdefine display compositions for pages of a multi-page menu, and eachpage information includes one or more pieces of button information; eachpiece of button information is used to present a graphics object as onestate of a button material, so that each page of the multi-page menu ispresented as an interactive display; the stream model information showswhether (i) the graphics stream recorded on the recording medium ismultiplexed with a video stream and to be supplied to a playbackapparatus or (ii) the graphics stream recoded on the recording medium isnon-multiplexed with a video stream and to be preloaded to a memory ofthe playback apparatus; and the time information is used for causing astate transition of the multi-page menu in accordance with a playbackproceeding of the video stream, the time information being valid onlywhen the stream model information shows that the graphics stream ismultiplexed with the video stream.
 2. The recording medium according toclaim 1, wherein: the time information includes a selection timeoutstamp and a user timeout duration; each piece of button informationincludes a SetButtonPage command that instructs, when a correspondingbutton material is changed to an activated state, to set one of thepages other than a first page as a current page; the selection timeoutstamp shows a time for automatically activating a button material on thecurrent page, so that a SetButtonPage command corresponding to theactivated button material is executed; and the user timeout durationshows a time for causing a state transition of the multi-page menu to afirst page only display state by changing the current page back to thefirst page.
 3. A playback apparatus for playing back a video stream anda graphics stream, the playback apparatus comprising: a video decoderoperable to decode the video stream to obtain a moving picture; and agraphics decoder operable to decode the graphics stream to obtain afirst page and a second page, wherein: the graphic stream includes thegraphic stream includes one or more interactive composition segments andobject definition segments, each object definition segment defining agraphics object; each interactive composition segment includes timeinformation, stream model information, and a plurality of pieces of pageinformation; the plurality of pieces of page information define displaycompositions for pages of a multi-page menu, and each page informationincludes one or more pieces of button information; each piece of buttoninformation is used to present a graphics object as one state of abutton material, so that each page of the multi-page menu is presentedas an interactive display; the stream model information shows whether(i) the graphics stream recorded on the recording medium is multiplexedwith a video stream and to be supplied to a playback apparatus or (ii)the graphics stream recoded on the recording medium is non-multiplexedwith a video stream and to be preloaded to a memory of the playbackapparatus; the time information is used for causing a state transitionof the multi-page menu in accordance with a playback proceeding of thevideo stream, the time information being valid only when the streammodel information shows that the graphics stream is multiplexed with thevideo stream; and the graphics decoder is operable to control a statetransition of the multi-page menu in accordance with a playbackproceeding of the video stream based on the time information.
 4. Theplayback apparatus according to claim 3, wherein: the time informationincludes a selection timeout stamp and a user timeout duration; eachpiece of button information includes a SetButtonPage command thatinstructs, when a corresponding button material is changed to anactivated state, to set one of the pages other than a first page as acurrent page; the selection timeout stamp shows a time for automaticallyactivating a button material on the current page, so that aSetButtonPage command corresponding to the activated button material isexecuted; and the user timeout duration shows a time for causing a statetransition of the multi-page menu to a first page only display state bychanging the current page back to the first page.
 5. The playbackapparatus according to claim 3, wherein: the graphics decoder includes acomposition buffer operable to store one of the interactive compositionsegment, and a controller operable to decode the plurality of pieces ofpage information included in the interactive composition segment storedin the composition buffer; when the stream model information of theinteractive composition segment stored in the buffer shows that thegraphics stream is multiplexed with the video stream, the controller isoperable to discard the interactive composition segment stored in thecomposition buffer in accordance with the playback proceeding of thevideo stream; when the stream model information shows that the graphicsstream is preloaded to the composition buffer, the controller isoperable to retain the interactive control information regardless of theplayback proceeding of the video stream.
 6. A method for recording ontoa recording medium, the method comprising: generating application data;and recording the application data onto the recording medium, wherein:the application data includes a video stream and a graphics stream; thegraphic stream includes the graphic stream includes one or moreinteractive composition segments and object definition segments, eachobject definition segment defining a graphics object; each interactivecomposition segment includes time information, stream model information,and a plurality of pieces of page information; the plurality of piecesof page information define display compositions for pages of amulti-page menu, and each page information includes one or more piecesof button information; each piece of button information is used topresent a graphics object as one state of a button material, so thateach page of the multi-page menu is presented as an interactive display;the stream model information shows whether (i) the graphics streamrecorded on the recording medium is multiplexed with a video stream andto be supplied to a playback apparatus or (ii) the graphics streamrecoded on the recording medium is non-multiplexed with a video streamand to be preloaded to a memory of the playback apparatus; and the timeinformation is used for causing a state transition of the multi-pagemenu in accordance with a playback proceeding of the video stream, thetime information being valid only when the stream model informationshows that the graphics stream is multiplexed with the video stream. 7.A computer program used for enabling a computer to play back a videostream and a graphics stream, comprising code operable to cause thecomputer to: decode the video stream to obtain a moving picture; anddecode the graphics stream to obtain a first page and a second page,wherein: the graphic stream includes the graphic stream includes one ormore interactive composition segments and object definition segments,each object definition segment defining a graphics object; eachinteractive composition segment includes time information, stream modelinformation, and a plurality of pieces of page information; theplurality of pieces of page information define display compositions forpages of a multi-page menu, and each page information includes one ormore pieces of button information; each piece of button information isused to present a graphics object as one state of a button material, sothat each page of the multi-page menu is presented as an interactivedisplay; the stream model information shows whether (i) the graphicsstream recorded on the recording medium is multiplexed with a videostream and to be supplied to a playback apparatus or (ii) the graphicsstream recoded on the recording medium is non-multiplexed with a videostream and to be preloaded to a memory of the playback apparatus; thetime information is used for causing a state transition of themulti-page menu in accordance with a playback proceeding of the videostream, the time information being valid only when the stream modelinformation shows that the graphics stream is multiplexed with the videostream; in said decoding of the graphics stream, a state transition ofthe multi-page menu is caused in accordance with a playback proceedingof the video stream based on the time information.
 8. A method forplaying back a video stream and a graphics stream, the methodcomprising: decoding the video stream to obtain a moving picture; anddecoding the graphics stream to obtain a first page and a second page,wherein: the graphic stream includes the graphic stream includes one ormore interactive composition segments and object definition segments,each object definition segment defining a graphics object; eachinteractive composition segment includes time information, stream modelinformation, and a plurality of pieces of page information; theplurality of pieces of page information define display compositions forpages of a multi-page menu, and each page information includes one ormore pieces of button information; each piece of button information isused to present a graphics object as one state of a button material, sothat each page of the multi-page menu is presented as an interactivedisplay; the stream model information shows whether (i) the graphicsstream recorded on the recording medium is multiplexed with a videostream and to be supplied to a playback apparatus or (ii) the graphicsstream recoded on the recording medium is non-multiplexed with a videostream and to be preloaded to a memory of the playback apparatus; thetime information is used for causing a state transition of themulti-page menu in accordance with a playback proceeding of the videostream, the time information being valid only when the stream modelinformation shows that the graphics stream is multiplexed with the videostream; in said decoding of the graphics stream, a state transition ofthe multi-page menu is caused in accordance with a playback proceedingof the video stream based on the time information.